DETR是Transformer用于目标检测的开山之作。不同于传统Anchor Based多个Anchor回归一个目标框的"多对一"方案，其在损失计算阶段使用二分图匹配（使用匈牙利算法）来一对一地选择目标框进行损失。得益于query作为一种可学习的先验且其能与整个图像交互，不断修正自身的特点；query的数目能够远远小于Anchor的数目。少量的query为使用一对一匹配创造了条件，无需使用多对一增加正样本数目，即可缓解正负样本不平衡带来的问题。除此之外一对一匹配的使用进一步移除了为多对一不优雅的NMS后处理，帮助DETR实现了真正优雅的端到端。
下面结合代码分析一下这部分。
损失计算入口
# dert.py/forwardimages = self.preprocess_image(batched_inputs)
output = self.detr(images)if self.training:gt_instances = [x["instances"].to(self.device) for x in batched_inputs]targets = self.prepare_targets(gt_instances)# 损失计算入口loss_dict = self.criterion(output, targets)# 加权计算损失weight_dict = self.criterion.weight_dictfor k in loss_dict.keys():if k in weight_dict:loss_dict[k] *= weight_dict[k]return loss_dict## 损失计算```python
# matcher.py/HungarianMatcher/forward
# 源代码注释十分详细啊，首先并行计算batch_size个图像中的代价矩阵，在DETR中该代价/损失为
# query与target之间的 分类损失+回归L1损失+回归Giou损失
# 代价矩阵并行计算完成后拆分成每个Batch，传递到scipy库中实现的linear_sum_assignment函数中，其作用是在N个query和M个target中
# 寻找到一个一对一的匹配使得损失最小化@torch.no_grad()def forward(self, outputs, targets):""" Performs the matchingParams:outputs: This is a dict that contains at least these entries:"pred_logits": Tensor of dim [batch_size, num_queries, num_classes] with the classification logits"pred_boxes": Tensor of dim [batch_size, num_queries, 4] with the predicted box coordinatestargets: This is a list of targets (len(targets) = batch_size), where each target is a dict containing:"labels": Tensor of dim [num_target_boxes] (where num_target_boxes is the number of ground-truthobjects in the target) containing the class labels"boxes": Tensor of dim [num_target_boxes, 4] containing the target box coordinatesReturns:A list of size batch_size, containing tuples of (index_i, index_j) where:- index_i is the indices of the selected predictions (in order)- index_j is the indices of the corresponding selected targets (in order)For each batch element, it holds:len(index_i) = len(index_j) = min(num_queries, num_target_boxes)"""bs, num_queries = outputs["pred_logits"].shape[:2]# We flatten to compute the cost matrices in a batchout_prob = outputs["pred_logits"].flatten(0, 1).softmax(-1) # [batch_size * num_queries, num_classes]out_bbox = outputs["pred_boxes"].flatten(0, 1) # [batch_size * num_queries, 4]# Also concat the target labels and boxestgt_ids = torch.cat([v["labels"] for v in targets])tgt_bbox = torch.cat([v["boxes"] for v in targets])# Compute the classification cost. Contrary to the loss, we don't use the NLL,# but approximate it in 1 - proba[target class].# The 1 is a constant that doesn't change the matching, it can be ommitted.cost_class = -out_prob[:, tgt_ids]# Compute the L1 cost between boxescost_bbox = torch.cdist(out_bbox, tgt_bbox, p=1)# Compute the giou cost betwen boxescost_giou = -generalized_box_iou(box_cxcywh_to_xyxy(out_bbox), box_cxcywh_to_xyxy(tgt_bbox))# Final cost matrixC = self.cost_bbox * cost_bbox + self.cost_class * cost_class + self.cost_giou * cost_giouC = C.view(bs, num_queries, -1).cpu()sizes = [len(v["boxes"]) for v in targets]indices = [linear_sum_assignment(c[i]) for i, c in enumerate(C.split(sizes, -1))]return [(torch.as_tensor(i, dtype=torch.int64), torch.as_tensor(j, dtype=torch.int64)) for i, j in indices]