------------- Howto setup the Virtual File System (VFS) ------------- v 1.1 by BirdFlu * Table of Contents * --------------------- I. * Introduction II. * Configuration II.1 - Glossary II.2 - Basic Configuration II.2.1 main section II.2.2 profiles II.2.3 locations II.3 - Use Cases II.3.1 main game files II.3.2 "1.13 Mod" II.3.3 User Mods II.4 - Extension II.4.1 += Extension III. * Points of Discussion *********************************** I. Introduction *********************************** The Virtual File System (VFS) manages file handling and offers a unified view on the file system to the game at runtime. All files are accessible through a path inside the VFS. During the setup, files and directories can be mapped to a specified location in the VFS independent of its real location on the hard disc. Additionally to the mapping of real files and directories, also file archives can be mapped. Only SLF and uncompressed 7z archives are supported right now. The current layout of the VFS is based on the original Ja2 game data directory, i.e. the data directory is the root directory of the VFS. Further, there are directories that are defined by the original SLF archives (Anims, BigItems, Sounds etc.). These archives build the basic layer and on top of that come the files from the Data directory. So, whenever a file is accessed (with path/filename e.g. Tilesets/0/smguns.sti), the game looks in the Data directory first, and when the file cannot be found there, it looks in the according archive (tilesets.slf in this case). The 1.13 Mod adds another layer on top, the Data-1.13 directory, that behaves in the same way as the Data directory. That is, when a file is accessed, the game looks in the Data-1.13 directory first and only then continues with the previously described steps. The VFS extends this system even further. It externalizes the definition and configuration of these layers or profiles (as they are called in the VFS jargon) and allows for an unlimited number of them. Additionally, the 7z file (uncompressed) archive format is supported. Now the order of inclusion of the profiles and the actual files and directories lie in the hands of the user. Thus a proper configuration of the VFS is required. *********************************** II. Configuration *********************************** The configuration of the VFS is done in an ini file (vfs_config.ini). You can specify the filename in ja2.ini in key VFS_CONFIG_INI (section : Ja2 Settings). If you don't do it, it defaults to vfs_config.ini. But lets define some basic terms(bottom to top), first. II.1 Glossary =================================== Location : A location can either be a real directory or a file archive. All files inside these loactions are mapped into the VFS (recursively, i.e. contents of subdirectories are also mapped). The user can specify the path inside the VFS into which the external files are mapped. Profile : A profile is a named collection of locations. It is encouraged to define a new profile for every mod and it is also encouraged to keep the data of this mod "together". That is, the profile can define a profile root directory to which the location directories are appended during initialization. (Keeping the profile root empty gives you the possibility to stick together directories from very different places on your hard disc. A better solution is to define a new profile for non-related directories). Profile List (Stack) : Finally, the VFS is build from a list of profiles, where the list is proccessed from left to right, i.e. the leftmost will be included first and the rightmost profile will be included last (on the top of the stack). During file access, the profiles are processed in reverse inclusion order. A file search will start in the rightmost profile and will continue to the left. Profile mode : a profile can be defined as read-only (default) or as read-write. You cannot write to a read-only profile. As there are some files that have to be written (temporary file, savegames, logs, etc.) you have to define at least one profile as read-write. Usually this should be the top (rightmost) profile. Otherwise you could end up blocking a writable file with a read-only file. II.2 Basic configuration =================================== Lets start with a basic configuration. II.2.1 main section ----------------------------------- The ini file start with [vfs_config] PROFILES = profile1, profile2, profile3, ... The value of the key PROFILES is a list of profile names. For every element (profile) in this list another section must exist that has the format [PROFILE_$name], where $name is a value from the profile list (e.g. [PROFILE_profile1]. II.2.2 profiles ----------------------------------- [PROFILE_profile1] NAME = Some Arbitrary Name (probably a mod's Name) LOCATIONS = location1, location2, loaction 3 PROFILE_ROOT = Profiles\mod1 WRITE = TRUE Each profile definition must have a NAME key and also a LOCATIONS key. The value of NAME can be used inside the game to access the data of the specified profile. The value of LOCATIONS is again a list of location names. Additionally a PROFILE_ROOT path can be specified that will be prepended to all paths defined in the location sections. A profile can have the WRITE property (default is FALSE). At least one such profile must exist, as this is the place where temporary files will be saved. Usually the top profile will have the WRITE property. Obviously a file must be saved in a real directory, so one of the locations is supposed to have the TYPE = DIRECTORY. This directory should be mapped as root (MOUNT_POINT = ). The most simple case is to define only one location with the described properties. Thus the written temporary files will be saved in one place, instead of being scattered over multiple unrelated directories. II.2.3 locations ----------------------------------- [LOC_location1] TYPE = DIRECTORY PATH = MOUNT_POINT = [LOC_location2] TYPE = LIBRARY PATH = profiles/libs/archive.slf VFS_PATH = archive.slf MOUNT_POINT = A location section name is build from an element from the LOCATIONS list of the profile that is prepended with LOC_, e.g. [LOC_location1]. For every location the TYPE has to be specified and can either be DIRECTORY or LIBRARY. For a DIRECTORY you can specify the PATH (directory) and the MOUNT_POINT. The PATH value is appended to the PROFILE_ROOT path and decided where your files will be located in the real file system. With the MOUNT_POINT value you can influence the location of your files in the virtual file system. So, real path PROFILE_ROOT/PATH/local_dir/file.name will be mapped to /MOUNT_POINT/local_dir/file.name (accessible by just MOUNT_POINT/local_dir/file.name) A library is similar to a directory from the initialization point of view. But, additionally, it has another key, VFS_PATH, which is similar to the PATH (file) value. If the PATH value is not defined (= empty) or the specified file cannot be opened, then the VFS_PATH value is evaluated. In order for this to work, you have to have processed a location that contains the specified files. While this is a more complex configuration process, it gives you the possibility to integrate archives that are located in another archive themselves. II.3 Use Cases =================================== II.3.1 main game files ----------------------------------- The main use case is the setup of the main files for the game. ############## [vfs_config] PROFILES = Libs, Data ############## [PROFILE_Libs] NAME = Ja2 game libraries LOCATIONS = Anims, ..., Faces, ..., Tilesets, ... PROFILE_ROOT = Data [PROFILE_Data] NAME = Ja2 game files LOCATIONS = Files PROFILE_ROOT = Data ############## [LOC_Files] TYPE = DIRECTORY PATH = MOUNT_POINT = ############## [LOC_Anims] TYPE = LIBRARY PATH = anims.slf MOUNT_POINT = [...] [LOC_Tilesets] TYPE = LIBRARY PATH = tilesets.slf MOUNT_POINT = ############## So, basically, you have the profile that specifies the game libraries as described in (I.). Following is the profile with the Data directory. Since you need to have a writable profile, we define a user profile that will contain only user data (the other profiles contain only game data). [vfs_config] PROFILES = Libs, Data, UserProfile [Libs and Data as before] [PROFILE_UserProfile] NAME = User files LOCATIONS = PROFILE_ROOT = Profiles\UserProfile WRITE = true If LOCATIONS is empty and WRITE is set to TRUE, then a writable directory in PROFILE_ROOT will be initialized automatically. II.3.2 "1.13 Mod" ----------------------------------- Adding a mod is simple. We take the 1.13 Mod as an example. [vfs_config] PROFILES = Libs, Data, v113, UserProfile [Libs, Data and UserProfile as before] [PROFILE_v113] NAME = v1.13 LOCATIONS = datav113_dir PROFILE_ROOT = [LOC_datav113_dir] TYPE = DIRECTORY PATH = Data-1.13 MOUNT_POINT = The new profile for the Mod 1.13 is inserted into the profile list after the Data profile, but before UserProfile. So, whenever a file is accessed, it will be taken from the "Data-1.13" directory (when it exists there of course). If not, the game will search in the following profiles ("Data", then "Libs"). II.3.3 User Mods ----------------------------------- Instead of adding game data, the user can also add or replace the writable profile (user profile). Thus, he can try out a new mod (that uses the VFS system) without risking to overwite his savegames (and other temporary files). ################# [vfs_config] PROFILE = Libs, Data, v113, ExperimentalMod, UserProfile_exp ################# [Libs, Data, v113 as before] ################# [ExperimentalMod as in II.3.2] [PROFILE_ExperimentalMod] NAME = Files for experimental Mod LOCATIONS = ExpMod_Libs, ExpMod_Files PROFILE_ROOT = Profiles/ExpMod [LOC_ExpMod_Libs] TYPE = LIBRARY PATH = exp_mod.7z MOUNT_POINT = Tabledata [LOC_ExpMod_Files] TYPE = DIRECTORY PATH = ModFiles MOUNT_POINT = Interface ################# [PROFILE_UserProfile_exp] NAME = User file for ExpMod LOCATIONS = PROFILE_ROOT = Profiles\User_ExpMod WRITE = true (NOTE: the user directory has to exist, even if it is empty) Since you can define the filenames for you VFS configuration file in ja2.ini, you can create a couple of vfs config file and switch between them fairly easy ############# ja2.ini [Ja2 Settings] ;VFS_CONFIG_INI = vfs_config.main.ini ;VFS_CONFIG_INI = vfs_config.my_mod.ini VFS_CONFIG_INI = vfs_config.ExpMod.ini II.4 Extensions =================================== II.4.1 += Extension ----------------------------------- In an ini file it is possible to use the += operator, which appends a value to a previously set entry, making it a essentially list. This is possible, because internally a data structure (PropertyContainer) is used that contains a set of mappings, that in turn map two keys to a property [key1][key2] = property This data structure can be initialized from an ini file, which make 'key1' a section, 'key2' a key and 'property' the value of that section-key pair in that ini file. This data structure can also be initialized from multiple ini files, where the += operator appends a new value to an already defined entry. This extenstion is used in the configuration of the VFS, such that the definition of the VFS_CONFIG_INI value in in ja2.ini accepts a list of configuration files. As one value makes a one-valued list, this extension fits nicely in the system without requiring modifications to existing configuration files. So, a possible application of this extension could be the stand-alone configuration of a mod, where each mod brings an own configuration file that is merged with configuration files from other mods. Thus, it is easy to create a valid configuration for a larger number of mods, where otherwise for every mod combination an own configuration file would have to be created. And even for a small amount of (combinable) mods the total number of configuration files would increase very fast. For the default mod combination (Vanilla, 1.13) there would be two mod and one user configuration file ############# vfs_config.ja2.ini [vfs_config] PROFILES = SlfLibs, Ja2Data [PROFILE_SlfLibs] ... [PROFILE_Ja2Data] ... [LOC_...] ... ############# vfs_config.v113.ini [vfs_config] PROFILES += v113 [PROFILE_v113] NAME = v1.13 LOCATIONS = datav113_dir PROFILE_ROOT = [LOC_datav113_dir] TYPE = DIRECTORY PATH = Data-1.13 MOUNT_POINT = ############# vfs_config.user.ini [vfs_config] PROFILES += UserProf [PROFILE_UserProf] NAME = User Profile LOCATIONS = PROFILE_ROOT = Profiles\UserProfile WRITE = true ############## The actual combination of mods would be done in the file ja2.ini, where ############# ja2.ini [Ja2 Settings] VFS_CONFIG_INI = vfs_config.ja2.ini, vfs_config.user.ini would be the Vanilla game configuration, and the following ############# ja2.ini [Ja2 Settings] VFS_CONFIG_INI = vfs_config.ja2.ini, vfs_config.v113.ini, vfs_config.user.ini would be the setup for the "1.13" mod. Other mods can be added by simply entering its configuration file in the list ############# ja2.ini [Ja2 Settings] VFS_CONFIG_INI = vfs_config.ja2.ini, vfs_config.v113.ini, mod1.ini, mod2.ini, vfs_config.user.ini In order to avoid mixing savegames and temporary files of different mod combinations, a separate user profile directory can be used, where every user profile is defined in an own user configuration file. The user profile can be selected by adding the according configuration file at the end of the mod configuration file list. *********************************** II. Points of Discussion *********************************** TODO: ...