Post new topic    
Page 1, 2  »
Metal King Slime
Send private message
Data import/export tool 
 PostMon May 14, 2012 7:01 am
Send private message Reply with quote
I've finally finished the crude utility I promised (sorry BMR, Harlock!) for importing and exporting various data from RPG files to spreadsheets. It can handle items, enemies, heroes, attacks, and textboxes (but you should definitely use Custom's builtin textbox import/export, unless you really want them as a spreadsheet!) Advanced users only. It's not very friendly; I've decided that the approach is hopeless, and it needs to be rewritten.

It's written in Python using nohrio, and is available from my nohrio branch as 3rdparty/tabulate.py. There's a Windows binary here (4.1MB)

Here's the readme, which hopefully will convince enough people not to mess with this:
Quote:
A crude tool to import/export certain OHRRPGCE data lumps from/to csv files/spreadsheets. Tested with Libre Office and KSpreadsheet
Public Domain license.

Instructions:

* Upgrade your RPG file using Custom. Zenzizenzic may work; nightlies are recommended.
* Backup your game! This is not exactly thoroughly tested.
* Unlump your RPG file (using the unlump tool, Custom, or even nohrio!)
* Optionally edit the script to change the exported data fields
* Export data, eg.
tabulate.py --type items --export vikings.rpgdir items.csv
* Import the csv file into your favourite spreadsheet editor (you could import it directly into a different RPG file, but it would be so much simpler to just copy the lump)

You now have a spreadsheet with one row per record, column names in the first row, and one column per data field. You can delete columns, rearrange columns, add new columns with blank headers, and change the number of rows. When importing, the script ignores columns with blank names, and leaves unchanged any data fields for which the column is missing. Column names must be unchanged to be recognised, but they can be in any order.

* Read the relevant file format documentation page on the wiki:

Items: http://rpg.hamsterrepublic.com/ohrrpgce/ITM
Heroes: http://rpg.hamsterrepublic.com/ohrrpgce/DT0
Enemies: http://rpg.hamsterrepublic.com/ohrrpgce/DT1
Attacks: http://rpg.hamsterrepublic.com/ohrrpgce/DT6
Textboxes: http://rpg.hamsterrepublic.com/ohrrpgce/SAY

You'll need these to make sense of the contents of each cell. The data fields will be in the same order in the spreadsheet and on the wiki. For example, for items, "oobuse" is "When used out of battle", and a value of 6 in a cell in this column means attack 5. If you use a string which is longer than supported by the field, then it'll silently be cut short while importing. You'll always see fields for 64 elements, regardless of the number actually enabled in the game. Bitset fields are exported as a single column. Each bitfield cell contains the numbers of the set bits (ignoring obsolete or unused bits) comma separated, or in some cases names of the set bits instead. Examples of bit names are 'rename on add', 'Bob' (the name of a hero) or 'hero 3' (if the hero name is blank).

* Make changes
* Export the spreadsheet as a csv file. Make sure it's in the same format as originally exported: comma delimiters, quotes around strings if required.
* Before importing (into any unlumped RPG), make sure it has been upgraded to the latest file format, and there are at least as many records (items, heroes, etc) already in the game as there are in the spreadsheet: nohrio can not create new records.


EDIT: New version, updated README
Metal King Slime
Send private message
 
 PostMon May 14, 2012 7:36 am
Send private message Reply with quote
Marry me.

Hehe, but in all seriousness, this is awesome. Thanks a bundle, I'll download it now and give it a whirl. Cheers!

And thanks again!

Have I said how awesome you are for this? Hehe, cheers!
Being from the third world, I reserve the right to speak in the third person.

Using Editor version wip 20170527 gfx_sdl+fb music_sdl
Metal King Slime
Send private message
 
 PostMon May 14, 2012 11:45 am
Send private message Reply with quote
Got it working. Successfully imported around 30ish items into Legacy. Now I'm off to properly format my 100+ spreadsheeted items for importing, whee!

Thanks again, it works beautifully!
Being from the third world, I reserve the right to speak in the third person.

Using Editor version wip 20170527 gfx_sdl+fb music_sdl
Slime Knight
Send private message
 
 PostMon May 14, 2012 11:54 am
Send private message Reply with quote
Yay, thanks a ton, TMC! This is going to save me heaps of trouble. I really wasn't looking forward to defining ~1000 items by hand ;_;
SPELLSHARD: THE BLACK CROWN OF HORGOTH now COMPLETE! Grab it today!
Metal King Slime
Send private message
 
 PostMon May 14, 2012 12:06 pm
Send private message Reply with quote
Harlock Hero wrote:
I really wasn't looking forward to defining ~1000 items by hand ;_;


Most indeedy.
Being from the third world, I reserve the right to speak in the third person.

Using Editor version wip 20170527 gfx_sdl+fb music_sdl
Blubber Bloat
Send private message
 
 PostMon May 14, 2012 3:19 pm
Send private message Reply with quote
tool that can be used for mass item importing?
I assume this can be used for organization too, when I have a bunch more items to add to my game I'll have to try this out, as it seems you have a couple happy customers already.
dOn'T MiNd mE! i'M jUsT CoNtAgIoUs!!!
Play Orbs CCG: http://orbsccg.com/r/4r6x V
Metal King Slime
Send private message
 
 PostTue May 15, 2012 10:28 am
Send private message Reply with quote
Just finished importing 500+ items this afternoon. Mostly elemental (Fire, Cold, Electricity, Acid, Poison, Life, Death, Light, Dark, Holy, Profane, Magic) variants of weapons and armor. Almost any weapon can have two different elements tied to it, so that's quite a few different weapons. Those will prolly be tied to my crafting system once I get it up and working.

BMR is a happy OHRRPGCE-er today.
Being from the third world, I reserve the right to speak in the third person.

Using Editor version wip 20170527 gfx_sdl+fb music_sdl
Slime Knight
Send private message
 
 PostThu May 17, 2012 6:18 pm
Send private message Reply with quote
It's working great so far, but does anyone have any insight into how to handle the "equippable by" bits? I only get 8 of them showing up in my spreadsheet, despite having 13 defined heroes. I figure I must be misunderstanding something, and the definition of the ITM lump on the wiki isn't helping me much:

Quote:
66 INT * 4 Equippable by (41 bits, rest wasted)


Any help?
SPELLSHARD: THE BLACK CROWN OF HORGOTH now COMPLETE! Grab it today!
Metal King Slime
Send private message
 
 PostFri May 18, 2012 3:35 am
Send private message Reply with quote
As I said, bitset arrays are exported as individual bytes rather than bits. "INT * 4" is 8 bytes in total.

I can create a new format for the bitsets, will do that and create an exe next time I'm on Windows (really should have installed all my development tools in a VM instead...)

Since there are lots of bitsets I guess it would be better to export them as single cells, either as "0,1,1,1,0,1" or "1,2,3,5" or even "Frumpy,Olaf,Blam,Eric"
Slime Knight
Send private message
 
 PostFri May 18, 2012 7:20 pm
Send private message Reply with quote
Oh, okay, I get it now. Just the "jargon" was throwing me, which is my fault. Thanks for helping to explain it, TMC! This game may well have been close to impossible without your assistance.
SPELLSHARD: THE BLACK CROWN OF HORGOTH now COMPLETE! Grab it today!
Metal King Slime
Send private message
 
 PostSat May 26, 2012 5:52 pm
Send private message Reply with quote
Sorry for the delay; ran into bug stalemate and ended up forgetting about it for a week. I've finally uploaded a new version with real bitset support (brief description in the README). I was too lazy to name most bits, but you can try looking up the numbers on the wiki if you really want them.
Slime Knight
Send private message
 
 PostWed May 30, 2012 7:23 am
Send private message Reply with quote
Heya,

I'm running into an error that I can't make sense of. Everything was working okay, until I got to the point where I finally added in my item descriptions (column B). Now, when I try to import the .csv back into the .rpgdir, I get this error:

Code:
 Writing into records 0 to 1450 (out of 1451 existing)
Traceback (most recent call last):
   File "tabulate.py", line 342, in (module)
   File "tabulate.py", line 242, in csv2lump
   File "tabulate.py", line 126, in encoder
   File "nohrio\ohrstring.pyc", line 55, in set_str16
AssertionError


Had no issues whatsoever up to this point. Any idea what's wrong?
SPELLSHARD: THE BLACK CROWN OF HORGOTH now COMPLETE! Grab it today!
Slime Knight
Send private message
 
 PostWed May 30, 2012 7:28 am
Send private message Reply with quote
Just in case it matters, the item descriptions do contain extensive codes for extra characters and such; could this be messing with it in some way? Here's a typical example:

Code:
 Full Plate \156:50 \199:-10 : \175:0 \214:12 \215:12 \216:10


EDIT: Confirmed that the problem is in the item descriptions by temporarily clearing that entire column; it successfully imports when I do so.

DOUBLE EDIT: Okay, figured out what is causing the problem - it returns this error whenever there are more than 36 characters in column B, since that's the max length of the item description string.

HOWEVER - the only reason I'm exceeding 36 characters is because of the special symbols expanding into 4 characters ([helm symbol] -> \205, for example). In the engine, the strings would actually fit. Is there any way around this limitation?
SPELLSHARD: THE BLACK CROWN OF HORGOTH now COMPLETE! Grab it today!
Metal King Slime
Send private message
 
 PostWed May 30, 2012 11:42 am
Send private message Reply with quote
Actually, that's 42 characters.

By the way, if you're using the full 36 characters, make sure you're using a nightly since the 20th of April; before then only 34 to 35 characters were supported.

Opps, I didn't notice that it actually throws an error on strings that are too long rather than truncating as I said (there was an incorrect comment that said so, but I didn't notice it was wrong even though I fixed a bug in the assert! This is why source comments are a bad thing :P)

Edit: I completely forgot to mention that when switching to the new version, you'll have to delete most 'bitsets' columns in your old tables (including 'equippableby' in items), which are replaced by the new columns, except for the 'bitsets' columns for items which have completely disappeared since none of them are used.
Slime Knight
Send private message
 
 PostWed May 30, 2012 5:22 pm
Send private message Reply with quote
Yeah, I eventually realized I was going over 36 characters -_-;;

By that point, however, it was super-late, and I didn't come here to update, because I figured the fact that it bugged instead of truncating would still be of interest to you. In any case, I did fix my particular problem however! Thanks again for your help.
SPELLSHARD: THE BLACK CROWN OF HORGOTH now COMPLETE! Grab it today!
Display posts from previous:
Page 1, 2  »