Transforming Public Health Data Management: From Individual Use to Scalable Workflows with R

A Use Case from Argentina

María Cristina Nanton

m.nanton@buenosaires.gob.ar

Buenos Aires City Government | University of Buenos Aires

Carolina Mengoni Goñalons

cmengoni@buenosaires.gob.ar

Buenos Aires City Government | University of Buenos Aires

Cecilia Palermo

ceciliapalermo@buenosaires.gob.ar

Buenos Aires City Government | University of Buenos Aires

Information and Health Statistics Management Office

Ministry of Health of the City of Buenos Aires

25 people posing for a picture.

30+ data analysts, scientists and engineers
Interdisciplinary team
Data Policy, Data Science and Data Engineering subteams

Information and Health Statistics Management Office

What we do: Manage data and statistics pipelines

Users

Minister’s Office
Other government departments
Managers and professionals at healthcare facilities
Researchers
Citizens

Goals

Health policy and management decisions
Population health tracking and monitoring
Clinical decision making
Research

Information and Health Statistics Management Office

How we do it: design and mantain a data warehouse with processed data from our city’s Health Information System

The beginnings

2019 - March 2020

The beginnings: 2019 - Mar 2020

🎯 The Spark: How It All Started

🖥️ Recently implemented Electronic Health Record (EHR) in the city’s Health Information System…

🚫… no specialized team to extract and model that data for secondary uses

💡 Creation of the 10-person team that developed into our Office, the first ones to use that massive data to inform physicians, health centers and policy makers to make data-driven decisions

The beginnings: 2019 - Mar 2020

Initial Setup

RStudio Server
Server for code outputs
Secure, open source on-premises platform for file sharing: Owncloud
On-premises GitLab

Files and folders icons created by Freepik - Flaticon

The beginnings: 2019 - Mar 2020

Data sources

Csv icons created by surang - Flaticon

The beginnings: 2019 - Mar 2020

Products

Dashboards

Health center activity reports

The beginnings: 2019 - Mar 2020

Working with code

Library: agiseR

Executions:
- ✋ single-script manual runs
- ✋ Pipelines: Manual runs + script sourcing

Environments:
- ❌ No environment management… but almost all projects were solo projects

The beginnings: 2019 - Mar 2020

Working with code

Onboarding to code base:
- Relied on specific individuals per topic area, around loosely defined shifting conventions for data constructs… also, there was no turnover 😃

The beginnings: 2019 - Mar 2020

Case study: work for Primary Care Division

Basic reporting:

Number of patients and consultations per health center
Top problems (Problem-Oriented Medical Record)

To make our journey more concrete, we’re going to follow one thread throughout: our evolving work with the Primary Care Division. Our first dashboards for the Primary Care Division were simple but impactful. As shown in this screenshot, We used Flexdashboard to show data visualizations metrics like:

Number of patients and consultations per facility

Most common reasons for consultation

These indicators were also distributed via CSV and Excel reports, built manually and delivered periodically.

Because our EHR system was problem-oriented, we could identify and share the top health issues presented in each center—insightful for both managers and clinicians.

At this stage, the work was relatively static, updated occasionally, and focused on retrospective activity tracking.

Emergency and rapid development

Mar 2020 - 2022

Emergency and rapid development: Mar 2020 - 2022

What pushed us to the next level?

Emergency and rapid development: Mar 2020 - 2022

What pushed us to the next level?

Emergency and rapid development: Mar 2020 - 2022

City with 3M inhabitants
Massive circuits organized by the city:
- Case follow up and patients health monitoring
- Testing
- Vaccination

Emergency and rapid development: Mar 2020 - 2022

Data sources: data warehouse (🚀New!)

Emergency and rapid development: Mar 2020 - 2022

Products

Dashboards

Emergency and rapid development: Mar 2020 - 2022

Products

(New!) Indicators and metrics

Health tracking
Expense Recovery
Decision-Making
🔄 Weekly, Daily, and Multiple Daily Reports

This illustration is created by Scriberia with The Turing Way community. Used under a CC-BY 4.0 licence. DOI: 10.5281/zenodo.3332807

Emergency and rapid development: Mar 2020 - 2022

Working with code

Executions:
- ✋ Manual runs
- 🚀 Automation via cron jobs (with agiseR functions to try-catch error capturing and sending messages via email and telegram!)
- 🚀 Pipelines: at first, manual runs [sic] + script sourcing, organized in folders with clear versioning. Later, cron jobs 🤖

Environments:
- ❌ No environment capture… but almost all projects were (still) solo projects

Emergency and rapid development: Mar 2020 - 2022

Working with code

Onboarding to code base

Relied on specific individuals per topic area, around loosely defined shifting conventions for data constructs… ~~also, there was no turnover~~

Emergency and rapid development: Mar 2020 - 2022

Working with code

Onboarding to code base

Relied on specific individuals per topic area, around loosely defined shifting conventions for data constructs… ~~also, there was no turnover~~
😓 Anarchic, heterogeneous, one-to-one as team members left and new people were hired to replace them.
😓 Project-based organization. Clear need to structure technical assets documentation.

Emergency and rapid development: Mar 2020 - 2022

Working with code

Imagen: Divio

Emergency and rapid development: Mar 2020 - 2022

Working with code

Emergency and rapid development: Mar 2020 - 2022

Case study: work for Primary Care Division

Reports with a new Perspective:
- Before: ➕ counting patients who received care by facility
- Now: 🔬 Re-engaging prioritized populations who missed care during the pandemic
  - Detection of health conditions
  - Profiling individuals for follow-up

🧠 Long-term strategies required stable and scalable processes

🚀 First Data Mart table: sociodemographic registry

Maturity and consolidation

2022 - now

Maturity and consolidation: 2022 - now

What pushed us to the next level?

Internal drivers:

Fully remote team grown from 10 to 30+ people
Team members entered at different stages—pre-COVID in-person, virtual during COVID, and post-COVID—leading to differing work habits and collaboration styles.

This illustration is created by Scriberia with The Turing Way community. Used under a CC-BY 4.0 licence. DOI: 10.5281/zenodo.3332807

Maturity and consolidation: 2022 - now

What pushed us to the next level?

External drivers:

As the urgency of COVID-related projects subsided, medium- and long-term initiatives emerged, demanding the added value of seasoned analysts.

This illustration is created by Scriberia with The Turing Way community. Used under a CC-BY 4.0 licence. DOI: 10.5281/zenodo.3332807

Maturity and consolidation: 2022 - now

Our Ecosystem Today:

Fully grown data warehouse + data marts

This illustration is created by Scriberia with The Turing Way community. Used under a CC-BY 4.0 licence. DOI: 10.5281/zenodo.3332807

Maturity and consolidation: 2022 - now

Products

Complex phenotyping projects
Transversal use of Shiny apps & dashboarding
Solid data warehouse & data marts creating and monitoring processes

Database icons created by juicy_fish - Flaticon

Maturity and consolidation: 2022 - now

Working with code

Executions:
- ✋ Manual runs
- 🤖 Automation via cron jobs (with agiseR functions to try-catch error capturing and sending messages via email and telegram!)
- 🚀 Cron jobs data mart

Maturity and consolidation: 2022 - now

Working with code

Pipelines

This illustration is created by Scriberia with The Turing Way community. Used under a CC-BY 4.0 licence. DOI: 10.5281/zenodo.3332807

Maturity and consolidation: 2022 - now

Working with code

Pipelines: rtasker! (new tool 🚀)

Maturity and consolidation: 2022 - now

Working with code

Pipelines: rtasker! (new tool 🚀)

Maturity and consolidation: 2022 - now

Working with code

Environments

With 30+ team members, we have almost none solo projects…

Maturity and consolidation: 2022 - now

Working with code

Onboarding to code base: Now a key issue!

This illustration is created by Scriberia with The Turing Way community. Used under a CC-BY 4.0 licence. DOI: 10.5281/zenodo.3332807

Maturity and consolidation: 2022 - now

Working with code

Onboarding: internal Data Science team quarto website!

Maturity and consolidation: 2022 - now

Working with code

Onboarding to code base: practical guide

Maturity and consolidation: 2022 - now

Case study: work for Primary Care Division

🚀New information needs
- Monitoring and management of Healthcare Teams
- Outcome Indicators for Healthcare service lines (such as Health Checkups and Immunization Coverage)

Maturity and consolidation: 2022 - now

Case study: work for Primary Care Division

💪Data mart growth
- Stable and regular processes for modelling
  - Health Conditions, Health Coverage
  - Activity of Healthcare Teams and Primary Care Physicians
  - Medical Services Provided

Lessons learned

What have we learned?

Internal data engineering teams for project alignment

This illustration is created by Scriberia with The Turing Way community. Used under a CC-BY 4.0 licence. DOI: 10.5281/zenodo.3332807

What have we learned?

Invest in interdisciplinary teams

This illustration is created by Scriberia with The Turing Way community. Used under a CC-BY 4.0 licence. DOI: 10.5281/zenodo.3332807

What have we learned?

Free, open source pays off

Thanks!

Questions? Contact us at m.nanton@buenosaires.gob.ar