ÖSYM Annual Data Pipeline

Problem

ÖSYM publishes yearly university and program information in Excel files whose structure does not map directly to the application's relational model. The application also needs to preserve history, not simply replace one year's data with the next.

Constraints

A program's identity is not always stable across years. Departments can be renamed, discontinued, introduced, or published with changed identifiers, so an apparently convenient automatic match can corrupt historical relationships.

Approach

I built a pipeline that reads and normalizes the incoming workbook, compares it with existing records, and associates yearly program data where the relationship is sufficiently clear.

• Normalize the source into the application's expected structure.
• Compare current records with preserved historical data.
• Associate yearly program records rather than overwriting history.

Edge cases

Ambiguous or unusual records are surfaced in a human-readable review output. The workflow intentionally asks for a decision instead of silently making a risky assumption.

Result

The pipeline automated a previously manual process and saved approximately two working days in an annual update cycle while retaining a deliberate manual checkpoint for uncertain data.