In the early days of computing, data was stored in simple, flat files. Think of these as digital versions of paper documents in a filing cabinet. Each application, whether for student registration or library loans, had its own set of files, tailor-made for its specific tasks. This was a straightforward approach called a File Processing System (FPS), and for a while, it worked.

But as organisations grew and their data needs became more complex, the cracks in this system started to show. The very structure of FPS created a series of deep-rooted problems that made data difficult to manage, trust, and use effectively.

The most immediate problem was data redundancy, which is the unnecessary duplication of data. Imagine a university where the Registrar's Office, the Library, and the Finance Department all maintain their own student records. Each system would store a student's name, ID, and address in separate files.

While this might seem harmless, it's the root of a much bigger issue: data inconsistency. What happens when a student moves? They might inform the Registrar's Office, which updates its file. But if that change isn't manually communicated to the Library and the Finance Department, their records become outdated. Now, the university has three different versions of the student's address. Which one is correct?

This leads to what are known as 'update anomalies'. These are errors that occur when you try to insert, update, or delete data in a redundant system.

• Modification Anomaly: This is the address change problem we just discussed. To update a single piece of information, you have to find and change every single copy of it. Miss one, and your data is inconsistent.

• Insertion Anomaly: What if a new student enrols but hasn't yet borrowed a book? The Library might not be able to create a record for them because their system requires a 'book borrowed' field. The data can't be added because of an arbitrary system rule.

• Deletion Anomaly: Imagine a student is the only person to have ever borrowed a rare book. If that student graduates and their record is deleted from the library system, the record of the book ever being borrowed might be deleted too. The loss of one piece of data unintentionally causes the loss of another.

Another major flaw of FPS is data isolation. Because each application's files were designed independently, their formats were often incompatible. The Registrar's system might store dates as DD/MM/YYYY, while the Finance system uses MM-DD-YY. One file might be a simple text file, another a proprietary binary format.

This effectively traps data within application-specific silos. Getting a complete, unified view of a student becomes a massive technical challenge. A simple question like, "Show me all students with outstanding library fines and unpaid tuition fees" could require a custom program to be written just to read, convert, and merge data from two completely different file systems.

This isolation makes it nearly impossible for an organisation to perform effective, data-driven analysis. The information exists, but it can't be easily accessed or combined.

Combined, data redundancy, inconsistency, and isolation create a significant maintenance overhead. Every time a new report is needed, programmers have to write complex, custom code to extract and process data from various files.

Even a small change can have a ripple effect. If the university decides to add a new field, like a student's phone number, to its records, every single application that reads the student file must be located, modified, and re-tested. This process is slow, expensive, and prone to error. As the number of files and applications grows, the system becomes increasingly brittle and difficult to manage, preventing scalable and efficient operations.

Let's check your understanding of these structural problems.

These limitations made it clear that a new approach was needed. Storing data in isolated, application-dependent files was simply not sustainable for growing businesses. This paved the way for the development of database management systems.