Overview
Developed an AI pipeline that automates the detection of mitotic (cancerous) cells in whole slide images (WSIs) of tissue samples. The system trains on canine tissue before testing efficacy on human tissue, addressing the limitations of manual pathological review — a process that is costly, expertise-dependent, and prone to disagreement between pathologists.
Architecture
Detection Stage: Systematic image segmentation combined with a fine-tuned Faster R-CNN (ResNet50 backbone) to generate candidate patches from whole slide images.
Classification Stage: ResNet18 classifier filters candidates, with the last convolutional layer enabling GradCAM visualization for interpretable results.
Explanation Stage: A dual-encoder architecture (Xception + BERT) maps detected cells to medical text descriptions using vision-text embeddings, generating natural language summaries alongside visual heatmaps.
Tech Stack
- Deep Learning: PyTorch, TensorFlow/Keras
- Models: Faster R-CNN, ResNet50, ResNet18, Xception, BERT
- Explainability: GradCAM activation maps
- Deployment: FastAPI, Docker, AWS EC2 (p3.2xlarge GPU instances)
- Data: SQLite, Jupyter notebooks
Deliverables
- Trained model weights for detection, classification, and text generation
- Web application accepting WSI uploads and returning heatmaps, activation maps, and natural language summaries
- Containerized deployment pipeline for local and cloud environments
Team
- Artemio Rimando (ArcSpan)
- Shelly Jain (JPMorgan Chase)
- Gage Sowell (Teledyne-FLIR)
Sponsored by Samsung through FourthBrain’s program.