Soundtracks for BG:EE, SoD, BG2:EE, IWD:EE, PST:EE are now available
in the Beamdog store.
A couple of streams ago someone asked about making 2das files into an SQL database. While this was immediately turned down I still felt the need to say why I think this would be a very bad idea for many reasons.
What follows is a much simplified explanation, to give you an idea of what we are dealing with here. For a fuller explanation of SQL look at the wikipedia entry for SQL
for an entry level explanation.
For a start there is no such beast as an SQL database. Structured Query language (e.g. SQL) is a language for manipulating the contents of a relational database. As such it is normally part of a Relational DataBase Management System (RDBMS). A relational database is designed as a method to store very large amounts of data while minimising duplication of that data. Not only that, such databases need to be designed to enable the easy retrieval of the data that is needed to be seen at any time. A relational database consists of a number of tables (typically in the thousands). Each table contains (usually) many records. Each record consists of 2 or more data items meaning that a typical relational database holds millions of data items. It is for such databases that SQL was designed based on the concepts of mathematical Set Theory. In order to make the retrieval of the data more efficient the records are indexed.
Compare that with 2da files. These are plain text files with the game data stored in tables. These tables are organised in rows and columns. Each row is a record and contains a number of data items. Each data item is separated by one or more spaces from the next. They do not require any specialised software to access or edit them. Any plain text editor can be used to read, edit or even create new ones (if you are a masochist you can even use the windows notepad).
Given the above these are the reasons why I think it would be a very bad idea to convert 2das to a relational database.
Possible advantages (i.e. not guaranteed)
- It might be quicker but I don’t think speed is an issue with 2da data handling. The 64 bit version (currently in development) may well speed this up anyway.
- There are a few ( less than 30 I think) 2das that it could help with for efficient storage of the data. I personally think (see "Bad Fit" below) that an Object Oriented DB would be a better solution for those. Even a redesign of the 2das themselves would help in those cases. But backwards compatibility scuppers that.
- Overkill. This should be obvious from the amount of data used.
- Backwards compatibility. This is obviously lost.
- The timeframe. To convert to a relational database would mean that the way that the data is organised would need to be redesigned before any migration of that data took place. This in turn mitigates against any custom (i.e. user made) data equivalent to custom 2das.
- The learning curve. To get the most out of a RDBMS and SQL in particular you need to learn set theory before even thinking about learning the rest.
- Bloat. Given the need for indexes the size of an equivalent relational database is much bigger than the combined size of all the 2da files.
- Bad fit. Given that there is some duplication of data in 2da files a relational database is not an ideal solution. If a migration of the data was forced on me I would say that an Object Oriented DataBase (wikipedia link) would be a much better fit due amongst other things to inheritance. Still needs specialist software though.
- Specialist software is needed to access, edit and create entries in a relational database.
- Not K.I.S.S. (Keep It Simple, Stupid). 2da files are simple to create, manage and use. A relational database is not.
- Yet another language to learn and this one looks nothing like C++/C#/C.