Work

2025 to present · co-founder and CTO

FluenTea: a video classroom with an AI speaking coach

I co-founded FluenTea so that language schools can prove their students speak. Teachers run live classes in it, and an AI coach called ChaCha sits in, runs practice calls between classes, and turns what students say into feedback for the student, the teacher and the department. I was the only technical person on the team.

  • What I bought and what I built. I paid for real-time video, a hosted speech model and a managed database, and spent my own time on the coach's behaviour, the feedback pipeline and the school workflow. Concept to launch took three months.
  • Safety and testing. Every action the coach can take is listed in a manifest, and a test fails if the model can see a tool that is not on the list. Anything that reaches another person needs a confirmed second turn. Text that did not come from the learner is treated as data. A red-team suite feeds her text written to make her misbehave, a second model grades her real answers against a baseline per error type that each release has to match or beat, and I read a sample of real transcripts every week.
  • Running it alone. Deploys go through a test gate and a check of the live database schema against the code. After each incident in eighteen months I added a check that would have caught it. To test a forty-student class on one laptop I wrote a harness that holds forty participants in a single browser, and after one traffic stall I moved the analytics writes behind a Redis queue so that a class never waits on them.
  • Listening to users. The early interviews I ran myself, and the dashboard rebuilt from them lifted monthly active users 247 percent in four months. Since October ChaCha runs them. She calls students and teachers at the partner university, asks eight scripted questions in Spanish or English, by voice or typed, and the server counts the answers and closes the interview, so the model never decides that it is done. When three of the first eight people answered the eighty-word opener with "¿Qué?", I cut it to a greeting and one question. The first round produced eleven fixes, built within the week, and the same mechanism now checks in with new users on their second day.
  • Finding problems before anyone reports them. A nightly script plays a confused first-time user. It signs up through the real form, wanders the app on desktop and phone, and files what breaks. Microsoft Clarity shows me where people rage-click or give up. ChaCha's own transcripts are a complaint channel, so the weekly read looks for bugs and confusion as well as for her mistakes. And the team holds its meetings inside FluenTea, so we hit problems before a school does.
  • The rest of a founder's week, with help. I wrote the outreach to language schools in Spanish, French and English as AI-drafted sequences by email and LinkedIn, with a do-not-contact list and a stop on the first reply. I also wrote the workflow that turns an advisor's or teacher's suggestion into code: a plan with the tests specified, coding agents that build it in parallel, the full test suite, and my review before anything merges. That is how eleven interview items became code in a week.

We launched three months after the concept, have more than 16,000 registered users from 136 countries, and work with three universities.

Python · FastAPI · PostgreSQL · React · LiveKit · Kubernetes on GKE · Cloud SQL · Cloudflare · Capacitor · Stripe

The product tour, two minutes (October 2026).
Click a screenshot to enlarge it.
2021 to 2026 · applied scientist, Amazon Berlin

Computer vision and multimodal models on the retail site

In a Berlin research team that partnered with product teams across the company, I was technical lead on three projects that went to production in front of customers, with multi-million dollar impact from higher sales and fewer returns. Five papers at Amazon's internal computer vision conference, one of them shortlisted for best paper.

  • What is in a creator's photo or video (2023 to 2024). I benchmarked zero-shot and low-shot CLIP against a supervised baseline, designed the crowd annotation over 270 interests and 30,000 videos, and set audit-based thresholds at 90 percent precision. The model launched into Amazon's interest mapping and feeds two recommenders. Lead author of the internal paper, an oral and best-paper shortlist.
  • Generative video and 3D (2025). Feasibility studies ahead of product decisions, from frame selection for structure-from-motion and Gaussian splatting to video transitions and product swap with diffusion models and image editing with Flux Kontext.
  • Product sizes from images (2021 to 2023). I proposed treating it as text visual question answering, built the golden dataset with a five-person annotation team across 130 product types, productionised the model to backfill more than ten million products, and led the public dataset release. Door Desk award; D-Extract at WACV 2023.
  • Visual similarity for Alexa Shop the Look (2022 to 2023). Embedding retrieval with diversity filters, index coverage from 15 to 95 percent after tracing the pipeline to one attribute, and a composed-retrieval model for refinement. Launched June 2023.
  • Also at Amazon. Supervised four interns and interviewed for the team (17 loops in one year), organised the Berlin ML Workshop 2023 with 180 attendees, won a GenAI hackathon, and received the Amazon Berlin Champion of Diversity and Inclusion award.

PyTorch · CLIP and vision-language models · text VQA · embedding retrieval · diffusion models · crowd annotation · A/B testing

Architecture figure from D-Extract: product image split into patches and OCR text boxes, fed together into a transformer with an image classification head and a bounding-box classification head
From D-Extract (WACV 2023): image patches and OCR boxes go through one transformer to find the dimensions. Paper · Dataset.
2019 to 2021 · research staff member, IBM Research Almaden

Table extraction for enterprise documents

Most of the numbers in a business document sit in tables, and in 2019 machines were still bad at reading them. At IBM I worked on the models and on the tools that let a domain expert correct them and train their own.

  • GTE (Global Table Extractor): technical lead and main developer of an end-to-end system that finds tables on a page and recovers their cell structure using visual context. WACV 2021.
  • TableLab: a labelling interface on top of GTE, from concept to launch, where users correct detected tables and the corrections retrain the model for their documents.
  • SEM-TAB-FACTS: lead organiser of SemEval 2021 Task 9, fact verification over tables in scientific papers. Co-author of CORD-19 (ACL 2020 workshop) and of D2S, document-to-slide generation (NAACL 2021).
Framework figure from the GTE paper: table detection with a penalty loss and re-ranking, then a cell network chosen by whether the table has graphical lines
From GTE (WACV 2021): table detection and cell structure recognition in one framework. Paper.
TableLab interface: a scanned financial report with two detected tables outlined in green, and the recovered cell structure on the right with merge, split and move controls
TableLab: detected tables on the page, recovered structure on the right, ready to correct.
2014 to 2018 · PhD, University of Washington

Decoding behaviour from brains and video

I wanted to use machine learning to understand the brain, and the brain to improve machine learning. The data was hundreds of hours of continuous recordings from patients with electrodes on the surface of the brain, alongside video and audio of whatever they happened to be doing in the hospital room.

  • AJILE (Annotated Joints in Long-term ECoG): sensor-fusing deep learning models that decode and predict movement from neural recordings and video, in natural rather than experimental conditions. Oral at AAAI 2018.
  • Automated annotation. Founded the project to annotate naturalistic neuroscience data automatically with computer vision and audio models. It became a multi-year effort funded by NSF and DARPA, and parts of the dataset were released publicly.
  • Earlier. Decoding dreams from fMRI, automatic classification of fish behaviour, and brain surgery in mice to study PTSD, during undergraduate research and a DAAD scholarship in Germany.

The released dataset · Papers on Google Scholar

Model figure from AJILE: at each time step before a movement, a video frame and an ECoG window pass through separate CNNs, are fused, and an LSTM predicts the coming movement
From AJILE (AAAI 2018): video and brain recordings fused to predict a movement before it starts. Paper.