Company overview
Caliber I.D. is a Massachusetts medical technology company that designs imaging systems for looking at skin tissue at the cellular level. Dermatologists capture images of a lesion at the bedside, then upload them to the company’s VivaNet® telepathology system, where dermatopathologists examine them and make a diagnosis.
The challenge
Caliber I.D. wanted to offer diagnosis as a service. That meant it needed licensed dermatopathologists who could read its images accurately, and a way to prove they could.
The company asked for three things in one solution:
- A way for newcomers to learn to read the images from the ground up
- A reference library of difficult-to-diagnose lesions
- Ongoing training on new cases for pathologists already diagnosing
The usual approach would have been to build a separate test environment and record software simulations for every type of lesion. That is expensive, it goes out of date every time the software changes, and grading and tracking stay manual.
Our approach
Instead of simulating the software, we recommended training people inside it. We added xAPI tracking to the real VivaNet system, so learners practiced on real cases with the same tools they would use on the job, and every step could be tracked, graded, and reported.
- A pool of real cases. Caliber I.D.’s dermatopathologists picked about 1,000 studies from the VivaNet archive, annotated them, and ranked them from level 1 (easiest) to level 3 (hardest). A learner masters a study by diagnosing it correctly three times. Until then, it can come up again at random.
- A testing module in the existing LMS. We built an image interpretation testing module on the LMS Caliber I.D. already used, with instructions and an embedded instance of VivaNet. Learners launched it like any other course.
- Detailed tracking. The module sent xAPI statements to a learning record store for each launch, navigation step, study viewed, diagnosis, and report detail. We set up a cloud Learning Locker™ LRS for reporting and connected it to the LMS with a custom plugin, so learners signed in once and new activity data sat alongside their existing training records.
- Parallel development. Caliber I.D.’s developers extended the VivaNet API to serve training studies. We wrote a Swagger API specification for them to build and test against, so both teams worked at the same time without waiting for each other.
How it works
When a learner opens the module, VivaNet serves a random study from the annotated pool. The learner examines each layer of the lesion, writes a diagnosis and report, and submits it. The system grades whether they identified the lesion correctly and whether they recommended a biopsy. Then it shows the licensed dermatopathologist’s annotations, so every attempt reinforces the right answer.
What the data showed
Because every action was tracked, Caliber I.D. could answer questions that a quiz score never could:
- Which studies take longest to diagnose, a sign that more training content is needed or that the images could be captured better.
- Which studies are consistently misdiagnosed, a second check on both the training and the image quality.
- Which pathologists are the most accurate, to find strong candidates for the diagnosis service and extra training for those who need it.
- How people move through the VivaNet interface, to spot where the software itself could be easier to use.
Results
- Cost less than building software simulations
- No outside costs to update the training: Caliber I.D.’s own dermatopathologists add new lesion examples and types
- No separate training hardware: training studies live in the production database, flagged as training
- Automated grading of every diagnosis
- A fast way to find exceptionally accurate dermatopathologists for the diagnosis service
- A feedback loop that tells the training team which materials work