Download CLEAR-10/CLEAR-100

Download links for CLEAR-10 and CLEAR-100

CLEAR Statistics. Note that CLEAR10 only has 1st-10th bucket labeled.

CLEAR-10 HuggingFace Download Links:

• CLEAR-10 Trainset (1GB)

• CLEAR-10 Testset (200MB)

• CLEAR-10 Trainset & Unlabeled Metadata (3GB)

CLEAR-100 HuggingFace Download Links:

• CLEAR-100 Trainset (3GB)

• CLEAR-100 Testset (1.6GB)

• CLEAR-100 Trainset & Unlabeled Metadata (13GB)

Instructions for downloading unlabeled images:

1. Download the CLEAR-10/-100 Trainset & Unlabeled Metadata.

2. Run the below Python script in the downloaded folder.

5KB

download_all_images.py

Python script for downloading the unlabeled images from YFCC100M metadata.

Folder Structure Preview:

root/
|   class_names.txt (each line is a class name)
|   labeled_metadata.json (with paths to labeled images' metadata)
|   all_metadata.json (with paths to all images' metadata)
|   download_all_images.py (for downloading unlabeled images)
└───labeled_images
|   └───1
|   |   └───computer
|   |   |   |   235821044.jpg
|   |   |   |   ...
|   |   └───camera
|   |   |   |   269400202.jpg
|   |   |   |   ...
|   |   └───...
|   └───2
|   └───...
└───labeled_metadata
|   └───1
|   |   |   computer.json (dict: key is flickr ID, value is metadata)
|   |   |   camera.json
|   |   |   ...
|   └───2
|   └───...
└───all_metadata
│   │   0.json
|   |   1.json
|   |   ...
└───features
|   └───moco_b0
|   |   |   features.json (with paths to labeled images' features files)
|   |   |   state_dict.pth.tar (a copy of the state_dict file)
|   |   └───1
|   |   |   |   computer.pth (dict: key is flickr ID, value is torch tensor)
|   |   |   |   camera.pth
|   |   |   |   ...
|   |   └───2
|   |   |   |   ...
|   |   └───...

Loading the pre-trained MoCo model on 0th bucket

• Here is a short example script for loading the state_dict.pth.tar under features/moco_b0 folder:

'''
Load pretrain model for image features
'''
import torch
from torchvision.models import resnet50

def load_model(state_dict_path='./features/moco_b0/state_dict.pth.tar'):
    model = resnet50(pretrained=False)
    model.fc = torch.nn.Identity()
    state_dict = torch.load(state_dict_path)
    model.load_state_dict(state_dict)
    for p in model.parameters():
        p.requires_grad= False
    model.eval()
    return model

Good to know: You can also download CLEAR-10 / CLEAR-100 on Avalanche with a few lines of code!