request for comment

hi guys, i began posting at http://www.ambrosiasw.com/forums/index.php...5&start=275 but thought i 'd consolidate it in the developer forum instead.

Quote

i'm still working on an engine that can read EVO plugs among a variety of other data formats. i switched languages to objective-c using the cocoa framework. one of the cool features i am working on atm is AI defined by plists. this will allow you guys to create and modify ships AI to be more EV like, as i will probably only create a couple myself and apply them to all ships. i know a few TC developers have wanted to create their own custom AI's, so i thought this would be a nice feature.

i'm still looking for other games that have data files, that i can write plugins, for my engine to read. they can be top down(EV, c&c, warcraft, diablo, etc), side scrolling/platform(flashback, ferazel, prince of persia, etc) and probably other types. the sky is almost the limit, but please no 3D models, only sprite based. thankyou.

my engine uses a module system for reading various types of data files.
ie. i am in the process of completing a module for reading the EVO data files and plugins. i have also started one for the original EV and a few old 68k games.
this allows me to write one engine, but have lots of data to test it with.

i would also like to define my own data/plugin files so people who like modding can easily and quickly make content for the engine, without being limited by another games data structure. i have a few basic ideas, but having never created anything more than a few hacks for EVO i really do not have any great ideas. this is why i come to you. i'd like to ask you guys, what are the good and the bad experiences you have had with plugin creation? ultimately, i'd like to get rid of all the bad things

for example, i think i recall from a few separate posts that resource ID numbers give people some headaches. why is this?
another issue is graphical editors. are these a good idea or is a data file expander/compressor a better option?
(by expander/compressor i mean, the content of a data file is constructed as files in a directory structure, like an xcode project, and then a utility does the work of putting all the pieces together into a data file)

it'd be great to hear from you guys on this topic. thanks a million.
tycho

This post has been edited by Tycho : 30 November 2008 - 08:06 AM

another idea i've had. plugin inheritance. if i download a plug 'X' and really like it except for one nagging element, for example a couple ships are way too overpowered, i can write a plugin 'Y' of my own which specifies it is based upon plugin 'X' but with a few minor changes. ie the couple ships speed or armour is decreased. likewise, if i wanted those ships completely removed, i could deprecate them. the engine would know these have been deprecated, so when they were required for a mission or something it would display some sort of notification to the player that another plugin has removed this element. this is where in game editing would be useful, so you could easily turn these ships back on and probably modify them some other way.

which brings me to in game editors. you could play the game in dev mode which would add play/pause, rewind, fast forward buttons(normal and dev mode have a record button). these buttons allow you to stop the game at any point and modify values on the fly. then you could rewind to the beginning of the mission and do it all over again with the new values. very handy for tweaking game dynamics without having to quit, edit and then restart the game.

if all this doesn't make you guys excited, i dunno what will. like i said, i need input of ideas to make this thing really great. so reply!!!

thanks,
tycho

I'll say I'm excited but I think designing a good data structure is not a simple task. If you want it done right it'll require a great deal of thought. I may offer some suggestions sometime later, just not right now

You did mention this before, and my response will be the same:

Very useful, and absolutely awesome, but a lot of work.

thankyou for the encouragement guy and templar

@templar98921, on Dec 1 2008, 07:19 AM, said in attn: plugin developers:

You did mention this before, and my response will be the same:

Very useful, and absolutely awesome, but a lot of work.

the work, i can do. it's just hard when i run into brick walls and have no reference to fall back upon. that's where i thought you guys would come in handy. i'm looking forward to your suggestions.

another concept i am flinging around is multi-genre. sort of like age of empires and spore. for instance, you could be flying around in your ship and then land on a planet and get out and walk around. or actually deliver supplies to a group of rebels in enemy territory. etc, etc, etc.

i believe there was a time when nova was going to have some sort of planet exploring functionality. i may be wrong, but i thought it was going to be a command and conquer/warcraft style. instead of ships, it was ground vehicles and squadrons of troops, etc. i only thought of this because i also recall an idea for an override plug that was pirate ships on the ocean, planets were islands, etc.

This post has been edited by Tycho : 02 December 2008 - 06:40 AM

no ideas? from anyone? /looks in mirror
hahaha

i got one. macFUSE. user space file system. it reached 2.0 the other day and is great. insanely great.
imagine mounting an ev plug, just like a disk image, on your desktop. brilliant.

Sorry, beat you to that one

Okay, data files. Since this is mac-only you could easily use bundles for the data 'files', with all the real files inside it. However if you want to compress the content you could just use a zip archive or something similar (maybe xar?) rather than creating a specialised archive format just for this purpose.
Cocoa can handle a variety of media formats so as long as people don't want to use something like ogg I guess you don't need to worry about what specific formats are used for images, audio and video.
As for the actual data I'd probably suggest a single xml file per bundle, containing arrays of ships, weapons etc. Text could be stored in a separate files, to allow localisation. Instead of IDs for all the objects you could have unique names - if you have two objects with the same name, one would override the other (the ability to override resources is important). One idea I've had before is inheritance: if you want, say, a variant of some base ship you could create your new ship in the data file and say it inherits from this other ship, then you just need to add your modified properties to it. If you later change something in the base ship you won't have to change all the variants as well.

Well that's as much as I'll say for now. Let me know your own thoughts/ideas

@guy, on Dec 18 2008, 03:42 AM, said in attn: plugin developers:

Sorry, beat you to that one

haha, that's cool. almost the same idea though.
i'm not thinking along the lines of a resource fork file system. while it would read/write the resource fork, it would not appear as a directory for pict's, etc., but more specific to the EV format. ie a directory for ships, in a particular ship directory would be the relevant picts and data. weapons would have picts/snds/data, etc, etc. if multiple weapons use the same snd, then it would merely be some kind of symbolic link.... which would then probably mean you would need a top level resource structure...hmmm.

@guy, on Dec 18 2008, 03:42 AM, said in attn: plugin developers:

Okay, data files. Since this is mac-only you could easily use bundles for the data 'files', with all the real files inside it. However if you want to compress the content you could just use a zip archive or something similar (maybe xar?) rather than creating a specialised archive format just for this purpose.

yep, that's what i'm thinking. could use FUSE here as well. zip the bundle, then "mount" the zip in memory.
do you think this would eliminate the need for an editor? the dev basically decides on whatever tools they want to use, then saves the files in the bundle.

@guy, on Dec 18 2008, 03:42 AM, said in attn: plugin developers:

Cocoa can handle a variety of media formats so as long as people don't want to use something like ogg I guess you don't need to worry about what specific formats are used for images, audio and video.

yeah, quicktime would handle media. if you wanted ogg, you'd have to install a qt codec.

@guy, on Dec 18 2008, 03:42 AM, said in attn: plugin developers:

As for the actual data I'd probably suggest a single xml file per bundle, containing arrays of ships, weapons etc.

yep, thinking of binary plists would be best(fastest?). but i don't really want to stop cross platform plugin creation. seems an odd consideration, but i think it's one worth thinking about.

@guy, on Dec 18 2008, 03:42 AM, said in attn: plugin developers:

Text could be stored in a separate files, to allow localisation.

good idea

@guy, on Dec 18 2008, 03:42 AM, said in attn: plugin developers:

Instead of IDs for all the objects you could have unique names - if you have two objects with the same name, one would override the other (the ability to override resources is important).

i agree. ID's are old hat. the file's path would be enough to use as a unique identifier. and since the macos will not allow you to have 2 files with the same name in the same directory, it's perfect!(and no, i'm not considering case-sensitive hfs :D)

@guy, on Dec 18 2008, 03:42 AM, said in attn: plugin developers:

One idea I've had before is inheritance: if you want, say, a variant of some base ship you could create your new ship in the data file and say it inherits from this other ship, then you just need to add your modified properties to it. If you later change something in the base ship you won't have to change all the variants as well.

@guy, on Dec 1 2008, 03:44 AM, said in attn: plugin developers:

another idea i've had. plugin inheritance.

i didn't really make myself clear, but inheritance would work inside plugs as well as across multiple ones. yep, i think we're on a winner here.
i keep thinking of a sprite map with no colour. you could easily create a fleet of them of different rank/version by inheriting a different coloured mask on them. obviously this would reduce plug size considerably(if made correctly).

thanks for your ideas guy. they really helped. i have a much clearer idea of how to progress now.

more ideas are always welcome!

@tycho, on Dec 18 2008, 05:43 PM, said in attn: plugin developers:

if multiple weapons use the same snd, then it would merely be some kind of symbolic link....

Or hard link? Anyways, I still don't think the benefits of such a thing would be worth the effort, at least if it was to be a special "EV-plug FS". Of course if the format is just zip then it's up to the user how they manage that archive and they are free to use FUSE if they so desire

@tycho, on Dec 18 2008, 05:43 PM, said in attn: plugin developers:

yep, that's what i'm thinking. could use FUSE here as well. zip the bundle, then "mount" the zip in memory.
do you think this would eliminate the need for an editor? the dev basically decides on whatever tools they want to use, then saves the files in the bundle.

If it's all just zip and xml then obviously you don't need an editor but editing the xml will be no better than ResEdit templates. There are no bounds on the possible features of an editor - imagine a full-blown plugin IDE complete with collaboration tools including subversion or git
But you're right, it does mean editing can be done without any special tools so you wouldn't need to focus on writing an editor of some sort.

@tycho, on Dec 18 2008, 05:43 PM, said in attn: plugin developers:

yep, thinking of binary plists would be best(fastest?). but i don't really want to stop cross platform plugin creation. seems an odd consideration, but i think it's one worth thinking about.

I haven't dealt with reading plists within a program before but surely the cocoa libraries wouldn't care whether they were text or binary, right? It'd simply be up to the plug dev as to which format they save their plists in.

@tycho, on Dec 18 2008, 05:43 PM, said in attn: plugin developers:

i agree. ID's are old hat. the file's path would be enough to use as a unique identifier. and since the macos will not allow you to have 2 files with the same name in the same directory, it's perfect!(and no, i'm not considering case-sensitive hfs :D)

Oh, so you're thinking one file per resource? I just thought it might be simpler to use only one file with arrays of resources. Anyway, it would work fine that way though you would only need the filename (minus the extension) as the identifier rather than the whole path (you wouldn't care if you had both outfit/BigGun.plist and weapon/BigGun.plist as they're different resource types). Remember you'll be using this identifier when one resource references another and you don't really want to be typing in long paths all the time

@tycho, on Dec 18 2008, 05:43 PM, said in attn: plugin developers:

i didn't really make myself clear, but inheritance would work inside plugs as well as across multiple ones. yep, i think we're on a winner here.

Ah, yeah I didn't quite understand what the purpose of plug-in inheritance was as opposed to simply overriding or inheriting from single resources. Say plug X contains a ship with identifier "SuperShip" then you make plug Y with "SuperShip" which will override plug X's "SuperShip" according to the alphabetical ordering of the plugs (the same way EV currently works but with these identifier names rather than IDs).

@tycho, on Dec 18 2008, 05:43 PM, said in attn: plugin developers:

i keep thinking of a sprite map with no colour. you could easily create a fleet of them of different rank/version by inheriting a different coloured mask on them. obviously this would reduce plug size considerably(if made correctly).

Heh, yeah I've thought of a similar thing myself but for Nova. Various different methods using alt sprites or running lights (which can be transparent) and PICT-based sprites of solid colours, allowing easy 'painting' of only parts of a ship (rather than the whole ship). I don't quite understand how inheritance would be relevant here though.

Data Sets

There are some places where are resource needs to reference a single, fixed resource of some other type (eg, a syst needs a govt, a weapon-type outf needs a weap). In this case you can simply enter the identifier for the desired resource.

There are other cases where a single resource is needed but there is no need for it to be fixed - it could be a different resource each time you encounter it (eg, a misn needs an 'availstel' spob, a spob needs a landing image). Then there are cases where multiple resources may referenced (eg, systs contain multiple ship types, ships have multiple different weaps). In these cases, you could simply enter a list of all the different identifiers desired.

...Or you could use a highly flexible approach and describe a complex data set using something like:
("Kestrel" | "Zephyr" | {"govt"="Federation" || ("techlevel">5 && "cost">1000000 && "cost"<3000000)}) & !"Raven"
Here we've specified a number of data sets and combined them together with | (union) and & (intersection). The first two are single identifiers and are equivalent to sets containing just that one element. The third is a query and contains all elements that match those conditions. The fourth is a set containing everything except the Raven. The result is a set containing the Kestrel, the Zephyr, everything belonging to the Federation and everything with tech level greater than 5 and cost between 1m and 3m, but excluding the Raven.

A set could be saved (as a dsët resource :p) and easily referred to at any time:
"Shuttle" | (@"SuperShips" & !@"CheatShips")
Here I've arbitrarily decided to use the @ symbol to indicate a set as opposed to an identifier for a single resource (this might not be very good syntax but it'll do fine for my example).

This allows us to do away with things like govt classes (or outfit classes). We can use data sets to finely control what weapons trigger weapon animations on what ships, what outfits trigger the 'keycarried' appearance of a ship, even what ships and outfits are available at a particular spob, rather than specifying a techlevel plus half a dozen specialtechs. Or allow a planet to have a random landing pic, which was so complicated in ARPIA 2.

This post has been edited by Guy : 18 December 2008 - 06:17 PM