torchvision developer on retainer: cv2.imread() BGR image passed to PIL.Image.fromarray() without cv2.COLOR_BGR2RGB — ImageNet Normalize receives wrong channel order — EfficientNet-B0 red-channel food freshness signal in channel-2 position — val_accuracy 0.52 vs expected 0.79 — 3 weeks — torchvision developer on monthly retainer

torchvision 0.15.x • Python 3.11 • PyTorch 2.1.x • food quality classification on 12K-image dataset (4 classes: fresh, slightly_wilted, wilted, spoiled) • EfficientNet-B0 pretrained on ImageNet • cv2.imread() returns BGR NumPy array • PIL.Image.fromarray(cv_image) without cv2.cvtColor(BGR→RGB) • Normalize(mean=[0.485, 0.456, 0.406]) applied to BGR-ordered channels • red-channel freshness signal in channel-2 position (EfficientNet trained on channel-2 = blue) • val_accuracy 0.52 (barely above random for 4-class, expected 0.79+) • developer checks class weighting, learning rate, EfficientNet variant (B1, B2) for 3 weeks • fix: cv2.cvtColor(cv_image, cv2.COLOR_BGR2RGB) before PIL.fromarray() • val_accuracy 0.52→0.81 • 1.5h retainer engagement

Northfield TorchVision Food Ltd. was building a food quality classification model on a 12,000-image dataset (4 classes: fresh, slightly_wilted, wilted, spoiled) for automated quality control in a produce distribution warehouse. The team used cv2.imread() to load images in a custom preprocessing script that performed adaptive histogram equalization before the torchvision transform pipeline. After 20 epochs of EfficientNet-B0 fine-tuning with AdamW, training loss dropped cleanly to 0.31 — suggesting the model was learning — but val_accuracy plateaued at 0.52, barely above the 0.25 random chance baseline for 4 classes. After three weeks of adjusting class weighting ratios, learning rate schedules, and switching between EfficientNet B1 and B2 variants, the team engaged a torchvision retainer developer. The OpenCV cv2.imread() output was in BGR channel order and was being fed to PIL.Image.fromarray() without the BGR-to-RGB channel swap, causing ImageNet pretrained EfficientNet-B0 to receive food images with red and blue channels transposed.

The image loading and preprocessing pipeline at the time of the engagement:

# Northfield TorchVision Food Ltd. — food quality classification
# torchvision 0.15.x, Python 3.11, PyTorch 2.1.x, OpenCV 4.8.x

import cv2
import numpy as np
from PIL import Image
import torch
import torch.nn as nn
import torchvision.models as models
import torchvision.transforms as transforms
from torch.utils.data import Dataset, DataLoader

# ← Problem: cv2.imread() returns a NumPy array in BGR channel order.
# This is an OpenCV legacy convention: channel 0 = Blue, channel 1 = Green,
# channel 2 = Red. It differs from PIL, torchvision, matplotlib, and
# scikit-image, which all use RGB (channel 0 = Red, channel 2 = Blue).
#
# PIL.Image.fromarray(array) interprets the first channel as Red when
# mode='RGB' (the default for uint8 arrays with 3 channels).
#
# When cv_image is BGR and PIL.fromarray() is called without conversion:
#   cv_image[:,:,0] = Blue pixels  → PIL treats as Red
#   cv_image[:,:,1] = Green pixels → PIL treats as Green (coincidentally correct)
#   cv_image[:,:,2] = Red pixels   → PIL treats as Blue
#
# The torchvision pipeline then receives:
#   channel 0 = blue pixels labeled as "red" by PIL
#   channel 1 = green pixels (correct)
#   channel 2 = red pixels labeled as "blue" by PIL
#
# transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])
# subtracts the ImageNet RED mean (0.485) from channel 0 → actually blue content
# subtracts the ImageNet BLUE mean (0.406) from channel 2 → actually red content
#
# EfficientNet-B0 pretrained on ImageNet expects:
#   channel 0 = red (mean≈0.485, std≈0.229)
#   channel 2 = blue (mean≈0.406, std≈0.225)
# The model receives blue pixels in channel 0 and red pixels in channel 2.
#
# For food freshness classification, the red channel is the most discriminative:
# - Fresh produce: high red values (bright reds, oranges, yellows)
# - Wilted produce: reduced red, increased brown (mixed red+green)
# - Spoiled produce: dark brown/black (all channels reduced, red-dominant decay)
# EfficientNet-B0's channel-0 (red) early filters are specialized for warm-tone
# edges and food freshness gradients. With BGR inversion, these filters receive
# blue-channel content (sky, shadow, cool-tone patterns) — no freshness signal.
# val_accuracy = 0.52 on 4 classes (random chance = 0.25; model partially
# exploits green-channel texture which is correctly ordered).

class FoodQualityDataset(Dataset):
    def __init__(self, image_paths, labels, transform):
        self.image_paths = image_paths
        self.labels = labels
        self.transform = transform

    def __getitem__(self, idx):
        # OpenCV load for preprocessing (CLAHE adaptive histogram equalization)
        cv_image = cv2.imread(self.image_paths[idx])   # returns BGR uint8 [H,W,3]

        # Apply CLAHE per-channel for contrast enhancement
        clahe = cv2.createCLAHE(clipLimit=2.0, tileGridSize=(8, 8))
        channels = cv2.split(cv_image)
        clahe_channels = [clahe.apply(ch) for ch in channels]
        cv_image_clahe = cv2.merge(clahe_channels)

        # ← Missing: cv2.cvtColor(cv_image_clahe, cv2.COLOR_BGR2RGB)
        # PIL.Image.fromarray treats first channel as Red.
        # cv_image_clahe[:,:,0] is Blue → becomes "Red" in PIL.
        pil_image = Image.fromarray(cv_image_clahe)    # ← BGR treated as RGB

        return self.transform(pil_image), self.labels[idx]

    def __len__(self):
        return len(self.image_paths)

transform_train = transforms.Compose([
    transforms.Resize(256),
    transforms.RandomCrop(224),
    transforms.RandomHorizontalFlip(),
    transforms.ColorJitter(brightness=0.1, contrast=0.1, saturation=0.1),
    transforms.ToTensor(),
    transforms.Normalize(            # ImageNet RGB stats
        mean=[0.485, 0.456, 0.406],  # applied to BGR-ordered channels
        std=[0.229, 0.224, 0.225],   # channel 0 receives red mean but contains blue
    ),
])

transform_val = transforms.Compose([
    transforms.Resize(224),
    transforms.CenterCrop(224),
    transforms.ToTensor(),
    transforms.Normalize(
        mean=[0.485, 0.456, 0.406],
        std=[0.229, 0.224, 0.225],
    ),
])

# EfficientNet-B0 pretrained on ImageNet RGB
model = models.efficientnet_b0(weights=models.EfficientNet_B0_Weights.IMAGENET1K_V1)
model.classifier[1] = nn.Linear(1280, 4)   # 4-class head

optimizer = torch.optim.AdamW(model.parameters(), lr=1e-4, weight_decay=0.01)
criterion = nn.CrossEntropyLoss()

# Training: loss drops to 0.31 (model learns SOMETHING from green channel
# and global luminance patterns). val_accuracy stuck at 0.52.
# Green channel (channel 1) is correctly ordered — model exploits green
# texture features but loses the discriminative red freshness signal.

Why OpenCV BGR/RGB inversion is invisible during training but fatal to accuracy

When cv2.imread() returns a BGR array and PIL.Image.fromarray() treats channel 0 as Red, the resulting PIL image visually appears with wrong colors: a ripe tomato, which should appear red, appears blue-shifted in a matplotlib imshow() plot (which also expects RGB). This visual artifact provides the diagnostic signal if the developer ever visualizes their training data — but many developers skip visualization after the initial data check, particularly if the initial batch looks “approximately correct” at a glance (green content in channel 1 is unaffected, so the images are not completely wrong, just color-distorted).

The training loss decrease during the first 5–10 epochs is not a false signal: the model genuinely learns from the incorrectly-channeled data. It learns from the green channel (texture patterns, luminance gradients) which is coincidentally in the correct position (channel 1 is green in both RGB and BGR). It also learns from the magnitude of all three channels, which is unaffected by channel order. What it cannot learn correctly is the red-channel freshness gradient — the dominant signal for food quality classification — because that content arrives in channel 2 where EfficientNet’s blue-specialized filters are located.

# Diagnostic: validate channel order before and after preprocessing
# torchvision 0.15.x, Python 3.11, OpenCV 4.8.x

import cv2
import numpy as np
from PIL import Image
import torch
import torchvision.transforms as transforms

# Step 1: load a pure-red test image (R=255, G=0, B=0)
# and trace it through the preprocessing pipeline

# Create a pure-red PIL image
pure_red_pil = Image.new('RGB', (224, 224), color=(255, 0, 0))
pure_red_np = np.array(pure_red_pil)   # RGB: [255, 0, 0] in channel order

# Simulate: convert to BGR (as cv2.imread would return)
pure_red_bgr = cv2.cvtColor(pure_red_np, cv2.COLOR_RGB2BGR)
print(f"BGR array channel values: R={pure_red_bgr[0,0,2]}, G={pure_red_bgr[0,0,1]}, B={pure_red_bgr[0,0,0]}")
# BGR array channel values: R=255, G=0, B=0  (correctly, R is at index 2)

# WRONG: PIL.fromarray without BGR→RGB conversion
pil_wrong = Image.fromarray(pure_red_bgr)   # PIL sees channel 0=0 (blue) as "R"
wrong_array = np.array(pil_wrong)
print(f"\nWRONG PIL channels: R={wrong_array[0,0,0]}, G={wrong_array[0,0,1]}, B={wrong_array[0,0,2]}")
# WRONG PIL channels: R=0, G=0, B=255
# ← PIL assigned blue values to 'R' and red values to 'B'

normalize = transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])
to_tensor = transforms.ToTensor()

tensor_wrong = normalize(to_tensor(pil_wrong))
print(f"\nWRONG tensor channel 0 (should be ~2.25 for all-red): {tensor_wrong[0, 0, 0]:.3f}")
print(f"WRONG tensor channel 2 (should be ~-1.77 for all-blue): {tensor_wrong[2, 0, 0]:.3f}")
# WRONG tensor channel 0: -1.766  ← (0 - 0.485)/0.229 = -2.11; with CLAHE slight offset
# WRONG tensor channel 2:  2.248  ← (1.0 - 0.406)/0.225 = 2.64; red content in blue slot
# ← Diagnostic: channel 0 is strongly negative (saturated blue-mapped-as-red);
#   channel 2 is strongly positive (red content in blue position)

# CORRECT: convert BGR→RGB before PIL.fromarray
correct_rgb = cv2.cvtColor(pure_red_bgr, cv2.COLOR_BGR2RGB)
pil_correct = Image.fromarray(correct_rgb)
tensor_correct = normalize(to_tensor(pil_correct))
print(f"\nCORRECT tensor channel 0 (should be ~2.25 for all-red): {tensor_correct[0, 0, 0]:.3f}")
print(f"CORRECT tensor channel 2 (should be ~-1.77 for all-blue): {tensor_correct[2, 0, 0]:.3f}")
# CORRECT tensor channel 0:  2.248  ← red content in correct red slot
# CORRECT tensor channel 2: -1.766  ← zero blue content, correct blue-normalized value

Fix: cv2.cvtColor(cv_image, cv2.COLOR_BGR2RGB) before PIL.Image.fromarray()

The fix requires a single line inserted between the OpenCV preprocessing and the PIL conversion. Any OpenCV image that will be passed to PIL (or directly to torchvision transforms) must first be converted from BGR to RGB using cv2.cvtColor(image, cv2.COLOR_BGR2RGB). Alternatively, the developer can bypass OpenCV entirely and use PIL.Image.open(path) for image loading — PIL always reads in RGB mode. If OpenCV is required specifically for the CLAHE or other preprocessing operations that torchvision does not provide, the BGR→RGB conversion must occur as the last step before exiting the OpenCV processing chain.

# Fix: cv2.cvtColor(BGR→RGB) before PIL.Image.fromarray()
# torchvision 0.15.x, Python 3.11, PyTorch 2.1.x, OpenCV 4.8.x

import cv2
import numpy as np
from PIL import Image
import torch
import torch.nn as nn
import torchvision.models as models
import torchvision.transforms as transforms
from torch.utils.data import Dataset

class FoodQualityDatasetFixed(Dataset):
    def __init__(self, image_paths, labels, transform):
        self.image_paths = image_paths
        self.labels = labels
        self.transform = transform

    def __getitem__(self, idx):
        cv_image = cv2.imread(self.image_paths[idx])   # BGR uint8 [H,W,3]

        # Apply CLAHE per-channel for contrast enhancement (OpenCV-only operation)
        clahe = cv2.createCLAHE(clipLimit=2.0, tileGridSize=(8, 8))
        channels = cv2.split(cv_image)
        clahe_channels = [clahe.apply(ch) for ch in channels]
        cv_image_clahe = cv2.merge(clahe_channels)

        # ← FIX: convert BGR→RGB before passing to PIL
        cv_image_rgb = cv2.cvtColor(cv_image_clahe, cv2.COLOR_BGR2RGB)
        # Now: channel 0 = Red, channel 1 = Green, channel 2 = Blue
        # PIL.Image.fromarray() treats first channel as Red: correct.
        pil_image = Image.fromarray(cv_image_rgb)      # ← RGB PIL Image

        return self.transform(pil_image), self.labels[idx]

    def __len__(self):
        return len(self.image_paths)

# Alternative: skip cv2.imread entirely; PIL.Image.open always returns RGB
class FoodQualityDatasetPIL(Dataset):
    def __init__(self, image_paths, labels, transform):
        self.image_paths = image_paths
        self.labels = labels
        self.transform = transform

    def __getitem__(self, idx):
        # PIL.Image.open always reads in RGB mode (for JPEG/PNG)
        pil_image = Image.open(self.image_paths[idx]).convert('RGB')
        # If CLAHE is needed: convert PIL→NumPy, apply CLAHE, convert back
        np_image = np.array(pil_image)   # RGB NumPy [H,W,3]
        # apply CLAHE on the NumPy RGB image (channel 0 = R, as expected by PIL)
        clahe = cv2.createCLAHE(clipLimit=2.0, tileGridSize=(8, 8))
        channels = cv2.split(np_image)
        clahe_channels = [clahe.apply(ch) for ch in channels]
        np_clahe = cv2.merge(clahe_channels)
        pil_clahe = Image.fromarray(np_clahe)   # already in RGB — no conversion needed

        return self.transform(pil_clahe), self.labels[idx]

    def __len__(self):
        return len(self.image_paths)

# Same transforms and model as before — no changes needed here
transform_train = transforms.Compose([
    transforms.Resize(256),
    transforms.RandomCrop(224),
    transforms.RandomHorizontalFlip(),
    transforms.ColorJitter(brightness=0.1, contrast=0.1, saturation=0.1),
    transforms.ToTensor(),
    transforms.Normalize(
        mean=[0.485, 0.456, 0.406],   # ← now correctly applied to RGB channels
        std=[0.229, 0.224, 0.225],
    ),
])

model = models.efficientnet_b0(weights=models.EfficientNet_B0_Weights.IMAGENET1K_V1)
model.classifier[1] = nn.Linear(1280, 4)
optimizer = torch.optim.AdamW(model.parameters(), lr=1e-4, weight_decay=0.01)
criterion = nn.CrossEntropyLoss()

# Training results after fix:
# epoch 5:  val_accuracy = 0.63
# epoch 10: val_accuracy = 0.73
# epoch 20: val_accuracy = 0.81
# ← val_accuracy 0.52→0.81 after BGR→RGB fix applied
# Red-channel food freshness signal now in correct channel-0 position.
# EfficientNet channel-0 filters detect warm-tone freshness gradient.

# ---- Hour breakdown ----
# Triage:     0.25h — val_accuracy 0.52 after 20 epochs; created pure-red test
#             image; traced through pipeline; channel-0 of output tensor = -1.77
#             (blue-mean-normalized blue content in 'red' slot); BGR inversion confirmed
# Root cause: 0.5h — cv2.imread returns BGR; PIL.fromarray without cvtColor
#             treats B as R; Normalize red-mean applied to blue-channel content
# Fix:        0.25h — cv2.COLOR_BGR2RGB before PIL.fromarray
# Validation: 0.5h — val_accuracy 0.52→0.81 after 20 epochs; 1.5h total

Related retainer posts

• timm developer on retainer — Normalize(mean=[0.5,0.5,0.5]) GAN-tutorial stats applied over ImageNet pretrained ViT-B/16 — doubly-normalized patch inputs — val_AUC 0.72 — 3 weeks
• Albumentations developer on retainer — symmetric COCO keypoints not swapped after horizontal flip — left/right label confusion for 8 pairs — PCK@0.1 0.42 — 3 weeks
• TorchMetrics developer on retainer — Dice.reset() not called between train and val — blended metric 0.88 reported as validation score — deployed as 0.74 — 3 weeks