Research

Journal and conference papers by members of our team, with colleagues at HUN-REN SZTAKI and Semmelweis University. Each summary links to the full paper.

Automated neonatal sleep positioning assessment by video monitoring and machine learning

Péter Földesy, Judit Varga, Ádám Nagy, Flóra Fehér, Zita Lilla Róka, Ákos Antal, Imre Gergely Jánoki, Miklós Szabó

How a preterm baby lies during sleep matters for muscle, bone and brain development, yet positioning is checked by hand, which takes time and varies between observers. In this proof-of-concept study, routine-care recordings of 50 stable preterm infants in closed incubators at two NICUs were used to train a pose estimator and machine-learning classifiers that recognize five sleeping postures and flag suboptimal positioning. Posture was classified with a macro F1-score of 0.80, and suboptimal positioning was predicted with an AUROC of 0.86: continuous, camera-based position checks between nursing assessments are feasible.

Sleep positionOpen access

Read the paper

2026

Medical & Biological Engineering & Computing, 64(7), 2677–2692June 2026

Automatic annotation to train ROI detection algorithm for premature infant respiration monitoring in NICU

Ádám Nagy, Péter Földesy, Imre Gergely Jánoki, Máté Siket, Zita Lilla Róka, Ákos Zarándy

Camera-based breathing measurement works better when the algorithm first finds the right region, the baby's chest or belly, but training a detector for that normally takes thousands of hand-labeled images. This paper creates those labels automatically from the periodic motion of breathing. The automatic masks agreed well with independent expert annotations (mean Dice score 0.735), and standard segmentation networks trained on them need no manual annotation, opening the way to large neonatal datasets.

BreathingOpen access

Read the paper

2025

Applied Sciences, 15(12), 6779June 2025

A Pilot Study: Sleep and Activity Monitoring of Newborn Infants by GRU-Stack-Based Model Using Video Actigraphy and Pulse Rate Variability Features

Ádám Nagy, Zita Lilla Róka, Imre Gergely Jánoki, Máté Siket, Péter Földesy, Judit Varga, Miklós Szabó, Ákos Zarándy

A newborn's activity level, behavioral state and sleep–wake cycle are judged today by direct observation, which takes staff time. This pilot study classifies infants' activity phases, adapted from the NIDCAP scale, with a stacked recurrent neural network (GRU) that works on features computed from the camera video and the patient monitor, and shows that pulse-rate variability improves the classification. The model was trained and evaluated on 108 hours of recordings from the NICU of Semmelweis University, Budapest.

Sleep & activityOpen access

Read the paper

2023

Engineering Proceedings (IEEE ECBIOS 2023), 55, 16November 2023

Neonatal Activity Monitoring by Camera-Based Multi-LSTM Network

Imre Gergely Jánoki, Ádám Nagy, Péter Földesy, Ákos Zarándy, Máté Siket, Judit Varga, Miklós Szabó

An infant's behavioral state, its level of arousal, affects sleep, brain development, bonding and feeding, but in preterm babies it is hard to judge and today calls for continuous observation by staff. This paper recognizes these states automatically from camera video with a multi-LSTM network that combines video actigraphy, a video-based breathing signal and other features, after separating care activity from undisturbed rest. It was trained and evaluated on 402 hours of all-day recordings of 10 babies at Semmelweis University.

Sleep & activityCare eventsOpen access

Read the paper

2021

Applied Sciences, 11(16), 7215August 2021

Continuous Camera-Based Premature-Infant Monitoring Algorithms for NICU

Ádám Nagy, Péter Földesy, Imre Gergely Jánoki, Dániel Terbe, Máté Siket, Miklós Szabó, Judit Varga, Ákos Zarándy

Camera-based vital-sign measurement had mostly been shown in ideal conditions: a calm baby and no one at the incubator. During care and active periods, motion makes the readings unreliable. This work detects and classifies those situations with high confidence, so that a single camera can monitor continuously, estimating pulse and breathing rate and recognizing medical interventions or very active periods, on ordinary CPU–GPU hardware. Evaluated on recordings from Semmelweis University.

Breathing & pulseCare eventsOpen access

Read the paper

IEEE Sensors Journal, 21(2), 2346–2352January 2021

Reference Free Incremental Deep Learning Model Applied for Camera-Based Respiration Monitoring

Péter Földesy, Ákos Zarándy, Miklós Szabó

A breathing-monitoring algorithm that needs no reference measurement and no prepared training set. It finds image areas with breathing-like periodic motion, collects examples while it runs, and incrementally trains a neural network to tell breathing apart from sudden or intense movement, adapting quickly as conditions change. It was validated on 24 hours of varied NICU recordings at Semmelweis University and on COHFACE, a public dataset of adults.

BreathingOpen access

Read the paper

2020

IEEE International Symposium on Circuits and Systems (ISCAS 2020), 1–5October 2020

Multi-Level Optimization for Enabling Life Critical Visual Inspections of Infants in Resource Limited Environment

Ákos Zarándy, Péter Földesy, Ádám Nagy, Imre Gergely Jánoki, Dániel Terbe, Máté Siket, Miklós Szabó, Judit Varga

Measuring a newborn's breathing and pulse with a camera is hard: the pulse signal is weak, breathing is irregular and babies move often, and the system has to run on limited embedded hardware. This paper presents lightweight neural networks for breathing monitoring, including a compact motion classifier, and a skin-segmentation approach to heart-rate measurement, evaluated on recordings from Semmelweis University.

Breathing & pulse

Read the paper

The papers describe research systems and results. NICOWL's product is in development; it is not CE-marked or FDA-cleared.