Small Objects in the Wild, Part 2: The Stock-YOLO Misfit
Part two of five. Part 1 covers the five geometric failures. Part 3 covers what actually moves the number. Part 4 covers tracking. Part 5 covers blur, sensors and deployment.
A stock YOLO is not bad. It is a well-engineered detector tuned for a specific distribution: COCO-like objects at 640 × 640. That distribution has almost nothing in common with an aerial, airborne or sports-ball one. Every default below is a reasonable choice for COCO and a poor one for you.
The numbers that frame the mismatch:
- 41.4% of COCO instances are “small”, but they occupy 1.23% of annotated pixels against 88.6% for large ones, and they appear in only 51.8% of images.
- 95.3% of SOD4SB bird instances are below 32 × 32 px.
- AI-TOD’s mean object size is 12.8 px.
- An object must be 48 px in a 4K frame to fill one P3 cell at imgsz 640.
Everything below was read from ultralytics/cfg/default.yaml, utils/tal.py, utils/loss.py, data/augment.py and the tracker configs at main = v8.4.126 on 22 August 2026, with comparisons against tags v8.2.0, v8.3.0 and v8.3.100. Values change between releases. The version you are on matters, and in one case it matters enormously.
1. The Pretraining Distribution
Model capacity follows pixels, not instance counts. Because small objects are 1.23% of COCO’s annotated pixels, the P5 branch of a COCO-pretrained checkpoint is heavily exercised, P3 weakly, and there is no stride-4 branch at all.
Moving to AI-TOD (12.8 px mean) or SOD4SB (median instance around 8 × 8 px, median relative size 0.002% of the image) is a three- to ten-fold shift in the size distribution — entirely outside the support of the pretraining data.
COCO initialisation is still the right starting point, because low-level features transfer and no in-domain corpus of comparable size exists for most applications. Add an in-domain pretraining stage where a large in-domain corpus does exist: the MVA bird challenges pretrain on a 47,260-image drone set, then fine-tune on the 9,759-image target set. For general scale, D-FINE-X reaches 59.3 AP when pretrained on Objects365 and fine-tuned on COCO under a 36-epoch early-stopped schedule.
One honest gap: no controlled A/B showing “VisDrone or DOTA initialisation beats COCO initialisation” surfaced during compilation. Treat domain pretraining as a well-supported pattern rather than a measured constant.
2. Head Strides P3/P4/P5, and No P2
At stride 8, a 6–16 px object spans 0.75 to 2 cells. At stride 32 it is sub-pixel. P2 configs exist — yolov8-p2.yaml, yolo11-p2.yaml — but they are not the default.
Measured on AI-TOD: FCOS 12.0 → 15.4 mAP with P2–P6, with AP on very tiny objects going 2.5 → 6.0. On VisDrone with YOLOv5 at 1536: mAP 28.88 → 31.03 and AP50 49.33 → 51.61, at 607 → 719 layers and 219 → 259 GFLOPs (+18%).
P2 is genuinely expensive: 25,600 extra positions at 640 px, dominating NMS and loss-assignment cost, with a matching VRAM spike. Switch to it if your targets are under about 20 px. Dropping P5 for pure aerial work is defensible engineering judgement — its receptive field exceeds any target — but no primary ablation isolating “remove P5” on aerial data surfaced, and the saving is in the backbone and SPPF rather than the head, since P5 is 400 positions out of 34,000.
The caveat that matters most: a P2 head alone is not enough. Faster R-CNN already has an RPN over P2–P6 and still scored AP on very tiny objects of exactly 0.0 on AI-TOD before its assignment was fixed. Adding resolution without fixing assignment does nothing for the smallest band.
3. imgsz=640 and Square Letterboxing
The long side is resized to 640, so a 4K frame is scaled by 0.167 and a 20 px object arrives as 3.3 px. Separately, rect: False means training pads every image to square, and on 16:9 input that is up to 44% of the tensor spent on grey.
640 is a genuinely good latency/accuracy point for COCO-like data, and it is what every published benchmark number uses, which keeps comparisons honest. For your data: train and infer at 1280–1536, set rect=True for non-square sources, and reduce batch accordingly. Combine with tiling — the two are complementary rather than alternatives.
Budget for the cost. 640 → 1280 is 4× FLOPs and roughly 4× activation memory, and the resulting tiny batch (2 at 1536) destabilises batch normalisation.
4. Task-Aligned Assignment: topk=10, α=0.5, β=6.0
This is the one most people never look at, and it is the most consequential.
The alignment metric is score^0.5 · IoU^6.0. Raising IoU to the sixth power is catastrophic at small scale. An 8 px box with 2 px of error has IoU 0.6, so IoU⁶ = 0.047. A 96 px box at IoU 0.9 gives 0.53. That is an eleven-fold difference in candidate ranking, driven entirely by object size.
Worse, target_scores are normalised to the maximum IoU per ground truth, so the classification target for a tiny object is capped by its achievable IoU. Its trained confidence saturates somewhere around 0.5–0.7 and never reaches 0.95.
Two downstream failures follow directly from that cap. The tiny object’s confidence sits below the default conf=0.25 at inference, which is why a demo looks far worse than the validation number. And the box and DFL losses are additionally scaled by the same alignment score (weight = target_scores[fg_mask].sum(-1)), so tiny objects get weaker localisation gradients too.
β = 6 sharpens assignment toward high-quality candidates, which is exactly right when high-quality candidates exist. When they cannot exist, it starves the target. Lower β to 2–4, and replace the IoU term with normalised Wasserstein distance or Dot Distance inside TaskAlignedAssigner.iou_calculation(). Reference implementations are Apache-2.0 in MMDetection forks (mmdet-rfla, mmdet-aitod) — but note that porting Apache-2.0 code into Ultralytics converts a permissive contribution into an AGPL obligation, not the reverse. See misfit 11.
5. The Sub-Stride Assignment Hole
This one is version-specific, and if you have ever concluded that sub-eight-pixel detection is impossible, read it before you believe yourself.
Through Ultralytics 8.3.x, select_candidates_in_gts simply tested whether an anchor centre lay inside the ground-truth box. The probability that the nearest P3 cell centre falls inside a w × h box is (w/8)(h/8). For a 6 px box that is 0.56, so about 44% of instances receive zero candidates. For a 4 px box it is 0.25, so about 75% receive zero. A ground truth with no candidate contributes no positive, no gradient, and is trained as background.
The fix has already shipped. In 8.4.x the assigner inflates sub-stride boxes before the in-box test: any ground truth narrower than the smallest stride is widened to 16 px, guaranteeing candidates. I verified this by diffing tal.py across tags.
So: upgrade, or backport the inflation. It costs nothing, changes no architecture, and removes the single largest silent failure for sub-eight-pixel targets. If you trained a sub-eight-pixel dataset on an older Ultralytics and concluded the task was impossible, retrain before trusting that conclusion.
No published measurement of what this bug cost exists — the 44% and 75% figures are derived here from the geometry. They are, however, hard geometry rather than an estimate.
6. Loss Defaults: CIoU + DFL with reg_max=16
Two separate problems, with box 7.5 / cls 0.5 / dfl 1.5 weights.
CIoU’s gradient vanishes for non-overlapping boxes, and its centre and aspect terms are dominated by the IoU term. For an 8 px box, where one pixel of error costs 22% IoU, the loss surface is a cliff rather than a slope.
DFL discretises each edge distance in units of one stride cell. At P3 one bin is 8 px, so representing a 6 px box means both sides land at roughly 0.375 bins and the softmax must interpolate inside bin 0–1. The relative localisation resolution is about 16× coarser than for a 96 px object at the same absolute precision.
DFL genuinely improves localisation on medium and large objects by modelling edge ambiguity, and CIoU is a good general-purpose box loss. The vendor evidently agrees about the small-object case, though: YOLO26 removes DFL entirely, describing it as reducing head complexity while preserving an unconstrained regression range.
Prefer a YOLO26-class head that has already dropped DFL, or swap the IoU term for NWD or SAFit. If you keep DFL, raising reg_max does not help — the bin width is set by stride, not by bin count. Note also that cls_pw: 0.0 means no class-frequency compensation, which on a long-tailed set like VisDrone leaves rare classes structurally starved.
7. Augmentation: mosaic=1.0, scale=0.5, close_mosaic=10
Mosaic itself does not shrink objects. It builds a 2 × imgsz canvas from four already-resized images and then crops back. The damage comes from the affine step: s = U[1−scale, 1+scale] = U[0.5, 1.5], so half of all training samples shrink every object by up to 2×, compounding with the letterbox shrink. Worst case at 4K/640: 20 px → 3.3 px → 1.7 px. Then translate=0.1 pushes tiny objects into the clipped border, where the box filter drops them.
Two things people commonly get wrong here. erasing: 0.4 is classification-only and does not affect detection training. And copy_paste is a silent no-op on box-only labels, because it calls drawContours on instance segments and therefore needs segmentation polygons. You need a custom transform.
Mosaic and scale jitter are a large part of why YOLO generalises well, and close_mosaic=10 is already a crude form of the staged-augmentation idea that challenge winners use deliberately. For small objects: scale=0.2, translate=0.05, close_mosaic=25–30, and add multi_scale=0.3. The MVA 2023 fifth-place entry used explicit staged augmentation — hard for the first half of training, light for the second — plus about 20% multi-scale sampling.
The arithmetic here is certain. The exact AP cost is not: no primary ablation isolating stock scale=0.5 as harmful on an aerial benchmark surfaced during compilation.
8. Postprocessing: conf=0.25, iou=0.7, max_det=300
Three traps in one line.
conf=0.25 removes exactly the detections whose confidence was capped by misfit 4. This is why a demo looks so much worse than the validation number — validation uses conf=0.001.
iou=0.7 sits on the wrong side of the tiny-object operating point in both directions at once. Two duplicate 8 px boxes 2 px apart have IoU 0.60 and both survive. Two genuinely distinct objects 3 px apart have IoU 0.45 and also both survive.
max_det=300 is fine at VisDrone’s mean of 52.9 objects per image but not in the crowded tail, and tiling multiplies it: 32 tiles × 300 = 9,600 candidates before the merge.
Then there is the hidden, worse version. pycocotools computes AP with maxDets=[1,10,100] — a 100-detection-per-image cap. Any COCO-API evaluation of a 300-object aerial frame caps recall at 33% regardless of how good the detector is. Tiny-object benchmarks raise this. Stock harnesses do not.
Set conf=0.05–0.10, iou=0.55, max_det=1000–2000, and raise your evaluation harness’s maxDets above 100. Better still, use an NMS-free head (YOLOv10, YOLO26) and delete two of the three problems.
9. Anchors and Autoanchor (v5/v7 Only)
Three failures stack here, which is a good reason not to start a tiny-object project from v5 or v7 at all.
YOLOv5’s autoanchor warns about objects under 3 px and then excludes boxes with both sides under 2 px from the k-means fit entirely, biasing the fitted priors upward precisely where you need them small. The matching metric (thr=4.0 on the width/height ratio) is scale-free, so a 3 px ground truth “matches” a 12 px anchor and best-possible-recall can read above 0.98 while every tiny match is a 4× scale error. And k-means on a distribution with a 3–10 px mode and 1 px quantisation is ill-conditioned — the code itself falls back with “switching strategies from kmeans to random init”.
The result is silently wrong priors plus a reassuring BPR number. Use an anchor-free version (v8 onward, or any DETR-family model). If you are pinned to v5 or v7, set anchors by hand from the actual size histogram and ignore BPR.
10. Ignore Regions Trained as Background
VisDrone’s ground truth uses category 0 for ignored regions, category 11 for others, and a score column where 0 means “do not evaluate”. The official toolkit excludes both from evaluation. Ultralytics’ own converter “skips regions marked as ignored” and drops the others category, which means those pixels become plain background in training. VisDrone has 12,371 ignored regions.
So every object inside an ignore region is trained as a hard negative, teaching the model to suppress exactly the appearance class it must detect elsewhere. The training signal actively fights the task.
Mask ignore regions out of the loss, or exclude those images, and enable class weighting for the long tail. Any dataset with an ignore convention has this trap — VisDrone, DOTA, WiderFace, and SODA’s rule of discarding anything above 2000 px².
No measured cost for this specific mistake surfaced. It is a training-signal contradiction rather than a subtlety, though.
11. Export, INT8, and the AGPL Problem
Ultralytics’ own TensorRT benchmarks show mAP50-95 going 0.37 → 0.33 and mAP50 0.52 → 0.47 under INT8, with 500 or more calibration images recommended. No per-size breakdown is published.
The mechanism to expect is that per-tensor activation scales are set by the range of high-energy large-object features, so low-amplitude tiny-object responses land in the bottom few quantisation levels, and the head’s sub-cell distinctions fall below one INT8 step.
An independent study of YOLO12 under static TensorRT INT8 supports that reading, and the pattern in it is the wrong way round for edge work: the nano model loses 7.2 mAP points against 3.1 for the x-large. Smaller backbones lose about 2.3× more accuracy to INT8 than the biggest one. Well-engineered post-training quantisation does far better — Hailo’s compiler holds YOLOv8n at −0.6 and YOLOv11n at −1.2 mAP — so this is a flow-quality problem rather than an inevitability. Part 5 goes through the numbers.
Then the licence. Ultralytics is AGPL-3.0 with a paid Enterprise alternative. The vendor’s licence page asserts that AGPL compliance requires publishing the complete corresponding source of the entire derivative work — including, in their reading, model weights — and that this applies to internal R&D, SaaS and API deployment, and embedded hardware alike. Under AGPL §13, network-interactive deployment triggers disclosure to users of the service, so “we only run it behind our API” is not a shelter. That reading of derivative-work scope is contested, but for planning purposes assume it.
This matters here more than usual, because every architecture change recommended in this article is textbook modification of an AGPL work. Either release under AGPL, buy the Enterprise licence, or build on Apache-2.0 alternatives: DEIM, D-FINE, RT-DETR, RF-DETR (N–L), YOLOX, MMDetection, PaddleDetection, with SAHI (MIT) and ByteTrack (MIT) alongside. YOLOv10 is also AGPL-3.0. And note D-FINE’s own warning that its Objects365-pretrained checkpoints may carry that dataset’s terms independently of the code licence — a licence trap with nothing to do with the code at all.
For the wider licensing picture, we covered YOLO commercial licensing separately.
The Remediation Ladder
Ordered by measured gain per unit of engineering effort. The ordering matters more than the list: applying step 3 before steps 1 and 2 wastes most of its benefit, because assignment cannot rescue pixels that were already destroyed.
| # | Change | Measured gain | Inference cost | Effort |
|---|---|---|---|---|
| 1 | Sliced training + sliced inference (20–25% overlap) | VisDrone AP50 +12.7 to +14.5; xView 2.1 → 20.4 | Linear in tile count (32× at 4K) | Low |
| 2 | Raise imgsz to 1280–1536, rect=True |
Prerequisite for everything else | 4× FLOPs at 640 → 1280 | Trivial |
| 3 | Swap the assignment metric (NWD / RFLA / DotD), lower β | AI-TOD +10.0 mAP; AP on very tiny objects 0.0 → 9.5 | Zero | Medium — code fork |
| 4 | Add a P2 (stride-4) head | AI-TOD +3.4 mAP; VisDrone +2.15 | +18% GFLOPs, 4.05× head positions | Config change |
| 5 | Fix the sub-stride assignment hole | 44–75% of 4–6 px targets go from zero to non-zero supervision | Zero | Trivial |
| 6 | Fix postprocessing (conf, iou, max_det, eval maxDets) |
Recovers detections already produced; removes a 33% recall cap | Zero | Trivial |
| 7 | Retune augmentation (scale=0.2, staged, close_mosaic=25+) |
Part of an MVA top-five recipe | Zero | Trivial |
| 8 | Hard-negative mining of sky and clutter | +3.7 mAP in 20 extra epochs (SOD4SB baseline) | Zero | Medium |
| 9 | Small-object copy-paste and oversampling | +9.7% / +7.1% relative AP_small on COCO | Zero | Medium — needs polygons |
| 10 | Honour ignore regions | Removes a direct training-signal contradiction | Zero | Medium |
| 11 | Ensemble + weighted box fusion | +3.9 AP50 (SOD4SB); multi-scale testing +1.27 mAP | N× the detector | Low |
| 12 | NMS-free head (YOLOv10 / YOLO26 class) | Removes the duplicate-box and detection-cap traps outright | Saves CPU, especially when tiling | Model swap |
Two rows deserve a note. Step 5’s gain is derived geometry rather than a published measurement, and steps 8 and 10 are unmeasured for cost — I have marked what is arithmetic and what is measured rather than blending them.
There is also a thirteenth idea with no numbers behind it: distilling from a tiled or high-resolution teacher. Ultralytics 8.4 ships a distill_model hook and the idea is obvious, but no measured deltas surfaced anywhere. It looks like the most promising unexplored lever in this whole area, because the teacher’s tiling cost is paid once at training time.
Stock Trackers Have the Same Problem, Only Worse
The tracker defaults fail arithmetically rather than statistically, which makes them easier to diagnose and more embarrassing to leave in place.
Ultralytics’ default tracker at 8.4.126 is tracktrack.yaml (track_high_thresh: 0.6, match_thresh: 0.7, gmc_method: sparseOptFlow). The BoT-SORT and ByteTrack configs use track_high_thresh: 0.25, match_thresh: 0.8, track_buffer: 30, fuse_score: True.
The Score-Fused IoU Gate
With fuse_score=True, the association cost is 1 − IoU·score, accepted when the cost is at or below match_thresh. At match_thresh=0.8 a match therefore needs IoU·score ≥ 0.2. A detection sitting at the first-stage floor (score 0.25) needs IoU ≥ 0.8, which for an 8 px box permits a maximum inter-frame displacement of 0.89 px.
Small fast objects cannot be associated at all out of the box. That is not a tuning problem; it is a gate that no plausible target motion can pass.
Be precise about which configs this applies to, though — only the ones that fuse. fuse_score exists in byte_tracker.py and bot_sort.py, and both bytetrack.yaml and botsort.yaml set it true. The 8.4.x default tracker, TrackTrack, does not fuse at all: tracktrack.yaml has no fuse_score key, and track_tracker.py builds a weighted multi-cue sum instead — an HMIoU distance (HMIoU = HIoU·IoU, with HIoU from vertical overlap), plus a weighted confidence difference, plus a ReID cosine term when embeddings exist, plus an angle term. Its plain gate at match_thresh=0.7 is HMIoU ≥ 0.3, and because HMIoU multiplies IoU by a vertical-overlap ratio, it is stricter than IoU alone for a target that moves vertically. The 0.6 and 0.7 values in that config are detection-confidence thresholds, not the fused gate.
The one forgiving path in the family is ByteTrack’s second association stage, which is IoU-only at threshold 0.5 with no score fusion. The paper is explicit that this is by design.
Set fuse_score=False, raise match_thresh, and — far more importantly — replace IoU with expanded IoU plus a normalised-distance penalty. Also check for upstream ByteTrack’s --min-box-area 100, which “filters out tiny boxes” by silently discarding anything under 10 × 10 px. On a small-object dataset that flag deletes the task.
Track Lifecycle and Camera Motion
Ultralytics sets max_frames_lost = track_buffer directly, whereas upstream ByteTrack scales it as int(frame_rate/30 · track_buffer). At 60 fps, track_buffer=30 is 0.5 s rather than 1 s. Separately, global motion compensation runs at half resolution (downscale=2) and tracks sparse features — the same texture your small targets do not have.
Set track_buffer to roughly 1.5 × fps for flicker-prone tiny targets, and pair the long coast with distance-based re-association. After five coasted frames IoU is zero, so an IoU-gated tracker cannot recover the track it was holding open for.
The DeepStream Equivalents
Three traps, all documented, and worth checking on any production pipeline:
tracker-widthandtracker-heightinternally rescale frames for tracking, and the canonical sample values are 640 × 384. On a 4K stream that turns a 12 px target into a 2 px target inside the tracker. This is the single most common misconfiguration here.minDetectorConfidenceis a second hidden confidence gate applied before tracking.probationAgewithholds new targets from output until they survive a probation window, which intermittently-detected tiny objects never do.
Raise maxShadowTrackingAge deliberately rather than by default, because every coasted frame with a constant-velocity Kalman filter on an erratic bird injects position error. If you are working through the DeepStream layer, our Savant versus DeepStream comparison covers where each sits.
Four Appearance-Free Changes, +11 SO-HOTA, No New Model
This is the canonical tracker-tuning result for tiny objects, from the MVA 2025 small-flying-object challenge. The detector never changes — YOLOv8-L detections are held fixed throughout:
| Change | Gain (SO-HOTA) |
|---|---|
| EMA smoothing of the velocity direction (α = 0.8), replacing noisy Kalman velocity | +3.72 |
| 2× box expansion before IoU, making the metric non-degenerate | +3.47 |
| Normalised centre-distance penalty replacing overlap as the primary similarity | +3.82 |
On an 8 px box, that 2× expansion raises the tolerated displacement from 5.3 px to 10.7 px. The full challenge result was 9.90 → 50.59 SO-HOTA on the private test set, a 5.1× improvement over the shipped baseline, with 44 M parameters. The tracker architecture was never the problem.
The One-Sentence Version
On tiny objects, roughly 85–95% of achievable tracking quality is decided by the detector and the association geometry. Almost none of it is decided by the tracker’s architecture. Spend your budget on tiling, resolution and the similarity function, and do not buy appearance re-identification at all — Part 4 explains why it is undefined at this scale rather than merely weak.
Part 3 takes the other side of the ladder: given that the defaults are fixed, which changes actually buy points, and in what order.