A Really Bad Idea

TarotRedhand

A couple of streams ago someone asked about making 2das files into an SQL database. While this was immediately turned down I still felt the need to say why I think this would be a very bad idea for many reasons.

What follows is a much simplified explanation, to give you an idea of what we are dealing with here. For a fuller explanation of SQL look at the wikipedia entry for SQL for an entry level explanation.

For a start there is no such beast as an SQL database. Structured Query language (e.g. SQL) is a language for manipulating the contents of a relational database. As such it is normally part of a Relational DataBase Management System (RDBMS). A relational database is designed as a method to store very large amounts of data while minimising duplication of that data. Not only that, such databases need to be designed to enable the easy retrieval of the data that is needed to be seen at any time. A relational database consists of a number of tables (typically in the thousands). Each table contains (usually) many records. Each record consists of 2 or more data items meaning that a typical relational database holds millions of data items. It is for such databases that SQL was designed based on the concepts of mathematical Set Theory. In order to make the retrieval of the data more efficient the records are indexed.

Compare that with 2da files. These are plain text files with the game data stored in tables. These tables are organised in rows and columns. Each row is a record and contains a number of data items. Each data item is separated by one or more spaces from the next. They do not require any specialised software to access or edit them. Any plain text editor can be used to read, edit or even create new ones (if you are a masochist you can even use the windows notepad).

Given the above these are the reasons why I think it would be a very bad idea to convert 2das to a relational database.

Possible advantages (i.e. not guaranteed)

• It might be quicker but I don’t think speed is an issue with 2da data handling. The 64 bit version (currently in development) may well speed this up anyway.
  
• There are a few ( less than 30 I think) 2das that it could help with for efficient storage of the data. I personally think (see "Bad Fit" below) that an Object Oriented DB would be a better solution for those. Even a redesign of the 2das themselves would help in those cases. But backwards compatibility scuppers that.
  

Disadvantages

• Overkill. This should be obvious from the amount of data used.
  
• Backwards compatibility. This is obviously lost.
  
• The timeframe. To convert to a relational database would mean that the way that the data is organised would need to be redesigned before any migration of that data took place. This in turn mitigates against any custom (i.e. user made) data equivalent to custom 2das.
  
• The learning curve. To get the most out of a RDBMS and SQL in particular you need to learn set theory before even thinking about learning the rest.
  
• Bloat. Given the need for indexes the size of an equivalent relational database is much bigger than the combined size of all the 2da files.
  
• Bad fit. Given that there is some duplication of data in 2da files a relational database is not an ideal solution. If a migration of the data was forced on me I would say that an Object Oriented DataBase (wikipedia link) would be a much better fit due amongst other things to inheritance. Still needs specialist software though.
  
• Specialist software is needed to access, edit and create entries in a relational database.
  
• Not K.I.S.S. (Keep It Simple, Stupid). 2da files are simple to create, manage and use. A relational database is not.
  
• Yet another language to learn and this one looks nothing like C++/C#/C.
  

TR

FreshLemonBun

I agree that 2da is a very simple interface for modding which some ppl still feel is too complicated. To add content a good starting step is usually copying a line for a piece of content similar to your idea, paste it at the bottom of the file, then update the row number. Any performance issue can probably also be solved while keeping the source files text based so performance isn't a great reason to change how people approach the files.

I think the appeal for some modders is that they want to give some other criteria like a tag and then receive the content rather than referencing everything by line number. That is when things become incompatible when two resources both use the same line number. It's not necessarily a bad idea but it does seem more complicated and would take a lot more work than simply typing in the line number in script functions and 2da references. It would also achieve basically the same thing the current system does but prevent ppl from breaking things by using incompatible mods. It probably wouldn't be entirely safe either because if content is identified by a creator given "tag" instead and two are the same you have the same problem. You could probably solve that too but I imagine it would complicate the process even more. It would be very different to copying a line, pasting it at the bottom, and increasing the line number.

Dark_Ansem

Isn't SQL the disaster that was the Dragon Age Origins Toolset?

At any rate, 2da is quite easy to use, just very limited.