Cross Domain Image Retrieval — OfficeHome Dataset

Transfer Learning · Fine-Tuning · Feature Extraction · FAISS Retrieval

---

Project Overview

Cross-domain image retrieval is the task of finding visually similar images across different visual domains. For example, given a sketch (Art domain) of a chair, the system should retrieve matching real photographs (Real World domain) and clipart/product images of the same category — even though the visual styles look completely different.

This project builds such a system on the OfficeHome dataset, which contains 15,588 images spread across 65 object categories and 4 distinct domains: Art, Clipart, Product, and Real World. The pipeline fine-tunes multiple pretrained CNN backbones for classification, extracts deep feature embeddings from the best model, compresses them using PCA, and indexes them using FAISS for fast cosine-similarity-based retrieval across domains.

---

Folder Structure

cross-domain-image-retrieval/
├── configs/
│   └── config.py                  # Hyperparameters, paths, experiment grid
├── src/
│   ├── dataset.py                 # OfficeHomeDataset, SubsetDataset, DataLoaders
│   ├── models.py                  # build_model / build_optimizer / build_scheduler / extract_backbone
│   ├── train.py                   # train_one_experiment / run_all_experiments
│   ├── feature_extraction.py      # extract_features / reduce_with_pca / build_faiss_index
│   ├── retrieval.py               # retrieve_numpy / retrieve_faiss / compute_map / compare_methods
│   └── visualize.py               # All matplotlib plots + console report
├── scripts/
│   └── main.py                    # End-to-end pipeline runner
├── results/
│   ├── all_exp_plot.png                           # Training loss vs epoch for all 6 experiments
│   ├── all_exp_train_vs_test_vs_val_accuracy.png  # Grouped accuracy bar chart (train/val/test)
│   ├── details.txt                                # PCA dimensions, feature sizes, dataset split info
│   ├── project_report.txt                         # Full project summary with all model results
│   ├── query_time.png                             # Query latency comparison: Numpy vs FAISS
│   └── retrieval_comparison                       # mAP@10 comparison: Numpy vs FAISS
├── .gitignore
├── README.md
└── requirements.txt

Dataset

OfficeHome — Venkateswara et al., CVPR 2017

Property	Value
Total images	15,588
Categories	65 (everyday objects: chair, laptop, fan, scissors, etc.)
Domains	Art, Clipart, Product, Real World
Train split (70%)	10,911 images
Validation split (15%)	2,338 images
Test split (15%)	2,339 images

Images cover diverse visual styles across the same object categories, making cross-domain retrieval a genuinely challenging problem.

---

Pipeline

Data Loading
     ↓
Model Fine-Tuning (6 experiments)
     ↓
Best Model Selection
     ↓
Feature Extraction (backbone → 2048-dim vectors)
     ↓
PCA Compression (2048 → 512 dims)
     ↓
FAISS Index Building
     ↓
Cross-Domain Retrieval + Evaluation (mAP@10)

Data Loading — OfficeHome images loaded across all 4 domains with a 70/15/15 train/val/test split. Train set uses augmentation (random horizontal flip, color jitter); val/test use only resize + normalize.

Model Fine-Tuning — Six pretrained CNN configurations are trained for 8 epochs. Only the last convolutional block and classifier head are unfrozen (selective fine-tuning). Best checkpoint per experiment is saved by validation accuracy.

Feature Extraction — The classification head is stripped from the best model, and all 15,588 images are passed through the backbone to produce 2048-dimensional L2-normalized feature vectors.

PCA Compression — Features are projected from 2048 → 512 dimensions using PCA, retaining 96.06% of explained variance.

FAISS Index Building — A FAISS IndexFlatIP (inner-product) index is built over all 512-dim compressed vectors for fast nearest-neighbour lookup at query time.

Retrieval — Given a query image, its feature vector is extracted, PCA-projected, and searched against the FAISS index. Results from the same domain as the query are excluded, enforcing true cross-domain retrieval. Performance is measured using mAP@10 (mean Average Precision at top-10 results).

---

Models & Experiments

Six configurations were fine-tuned and benchmarked:

#	Architecture	Optimizer	Dropout	Scheduler	Label Smooth
1	ResNet-50	AdamW	0.4	Cosine Annealing	0.1
2	ResNet-101	AdamW	0.5	Cosine Annealing	0.1
3	EfficientNet-B0	AdamW	0.4	Cosine Annealing	0.1
4	MobileNet-V3	Adam	0.3	Cosine Annealing	0.0
5	DenseNet-121	AdamW	0.4	Cosine Annealing	0.1
6	ResNet-50	SGD	0.4	Cosine Annealing	0.1

All models use ImageNet-pretrained weights. The custom classification head replaces the original FC layer: Linear → BatchNorm → Activation → Dropout → Linear(65 classes).

Loss Function

Cross-Entropy Loss with Label Smoothing is used for all experiments:

L = -sum [ y_smooth * log(p_predicted) ]

where y_smooth = (1 - ε) * y_true + ε / C
      ε = 0.1 (label smoothing factor)
      C = 65  (number of classes)

Label smoothing prevents the model from becoming overconfident on training labels, improving generalization to unseen domains.

Retrieval Similarity

Cosine similarity is used to rank retrieved images against a query:

sim(q, d) = (q · d) / (||q|| * ||d||)

Since all feature vectors are L2-normalized before indexing, cosine similarity reduces to the inner product — which is exactly what FAISS's IndexFlatIP computes, making the two approaches equivalent in quality but very different in speed.

---

Results

All Experiment Accuracies

#	Model	Train	Val	Test
1	ResNet50 + AdamW + Dropout0.4 + Cosine	98.98%	82.81%	83.88%
2	ResNet101 + AdamW + Dropout0.5 + Cosine	98.81%	84.09%	84.78%
3	EfficientNetB0 + AdamW + Dropout0.4 + Cosine	91.85%	76.35%	78.75%
4	MobileNetV3 + Adam + Dropout0.3 + Cosine	92.86%	77.20%	77.94%
5	DenseNet121 + AdamW + Dropout0.4 + Cosine	99.26%	83.23%	84.22%
6	ResNet50 + SGD + Dropout0.4 + Cosine	99.21%	84.56%	86.28% BEST

Best Model Details

Property	Value
Architecture	ResNet-50
Optimizer	SGD (lr=0.01, momentum=0.9, weight_decay=1e-4)
Dropout	0.4
Scheduler	Cosine Annealing
Label Smoothing	0.1
Train Accuracy	99.21%
Best Val Accuracy	84.56%
Final Test Accuracy	86.28%

Feature Extraction & PCA

Property	Value
Raw feature dimensionality	2048
After PCA compression	512
PCA variance retained	96.06%
Total vectors indexed	15,588

Retrieval Performance (mAP@10)

Method	mAP@10	Avg Query Time
Numpy brute-force cosine	0.9224	28.71 ms
FAISS IndexFlatIP	0.9251	1.71 ms
Speedup	—	16.8× faster

FAISS achieves marginally better mAP (0.9251 vs 0.9224) while being 16.8× faster than brute-force numpy search — making it the clear choice for any production or large-scale use.

---

Quickstart

# 1. Clone the repository
git clone https://github.com/<your-username>/prml-project.git
cd prml-project

# 2. Install dependencies
pip install -r requirements.txt

# 3. Set your dataset path in configs/config.py
#    DATA_DIR = "/path/to/OfficeHomeDataset"

# 4. Run the full pipeline
python scripts/main.py

---

Dependencies

• torch / torchvision — model training and feature extraction
• scikit-learn — PCA dimensionality reduction, brute-force cosine similarity baseline
• faiss-cpu — fast nearest-neighbour indexing and search
• matplotlib — training curves and result visualizations
• Pillow — image loading and preprocessing
• numpy — numerical operations

Future Improvements

• Replace CNN backbones with Vision Transformers (ViT) for richer global feature representations and better cross-domain generalization