Icoobook

The brief

Children’s reading apps tend toward two extremes: gamified to the point of distraction, or so passive that engagement collapses. The team set out to find a middle path — something that meets a child where they are, emotionally, and quietly invites them deeper.

I led the team as student PI, in collaboration with Tsinghua University.

The system

Three interlocking modules, each tuned to a different cognitive register:

• Linking Strokes — a reading-comprehension game where children connect strokes to advance the story.
• Magic Brush — children draw an answer to a prompt; the system converts the sketch into a story-world object.
• Reading Poetry — the centerpiece. The child reads aloud; a deep-neural-network model recognizes the emotional tone (joyful, melancholy, neutral, brave) from acoustic features (F0, energy, MFCC) and word embeddings (Word2Vec). The illustration that accompanies the verse re-paints itself to match the emotion.

The result is a story that listens.

Research & validation

I designed and ran an experience study with ~40 children, comparing Icoobook against a control reading app. Across attention, cognitive memory, and emotional satisfaction, Icoobook outperformed the baseline at statistically meaningful levels.

Recognition

The project was invited for showcase at ACM Multimedia, a top venue for multimedia computing research.

My contribution

End-to-end product design, interaction architecture, and the technical-research path that bridged the design team and the AI team at Tsinghua. Coordinated user research, ran the experience study, and shaped the final visual language for a child-friendly UI.