Looking Back to Look Forward — SENet[1] is one of the pioneer works that introduced the attention mechanism into the modeling of vision data. So what is attention mechanism? in mathematical terms, attention is all about “weighting” : that weights some features more over others. The larger the weight, the more influence they will contribute to…

Looking Back to Look Forward — XGBoost[1] and its successor LightGBM[2] are the de facto industrial standard for coping many real life machine learning problems that involve tabular data, like CTR prediction, weather prediction and fraud detection, just to name a few, even in the era of Deep Learning. Efficient Algorithm and system design share the…

Looking Back to Look Forward — The debut of Vision Transformer[1] confirms the effectiveness of handling vision data using Transformer[3] like architectures. The success of ViT is attributed to its long range relationship modeling ability that could capture the interactions of different image parts from the very early on. ViT sparks great enthusiasm among the Computer…

Looking Back to Look Forward — OCR[1] was mainly motivated by the fact that the label of a pixel is the category of the object that the pixel belongs to. So OCR augments pixels with Object Representations and at the same time learning the weights between the pixels and the object region representations. Logically, OCR augments pixels in three steps Construct soft region generator by learning from ground truth segmentation labels

One more step for zero shot segmentation — SAM tried to build a foundation model for segmentation, which consists of three interconnected parts. promptable segmentation task prompts can be fed into the model in various forms, e.g. points, box rectangles, free form texts or just masks. prompts and image labels guide the model to produce reliable and precise segmentation masks. segmentation model In…

Looking Back to Look Forward — Why did progress of self supervised learning like masked auto-encoder in Computer Vision lag behind that of Nature Language Processing(NLP)? Information Density Words are information dense, pictures are information redundant, some missing information can be inferred from pixels nearby So MAE employs a radically higher mask ratio like 75% to…

A Small Step Towards Open-Set Object Detection — Open-Set Detection leverages the learning of region-aware region embeddings, so that each region can be classified into novel categories in a language aware semantic space.[1] TBC Bullet points: The success of Grounding DINO [2] is attributed to the effective fusion of vision and language modalities from the very early on. Feature enhancement language-guided query selection

Looking Back to Look Forward — TBC CNN features are of shape LCHW, where L is the number of layers that are aggregated, HW tells the spatial size and C represents the number of channels. Dynamic Head [1] achieved state of the art when came out by combining scale aware attention(L dimension), spatial aware attention(HW dimension) and task aware attention(C dimension). Scale Aware Attention Spatial Aware Attention

Looking Back to Look Forward — Fine Grained Image Analysis requires the modeling of interactions between various parts of the image Bilinear CNN [1] can be seen as a special variation of the attention mechanism. TBC References [1] Lin, Tsung-Yu, Aruni RoyChowdhury, and Subhransu Maji. “Bilinear CNN models for fine-grained visual recognition.” Proceedings of the IEEE international conference on computer vision. 2015.