OHR preservation project -- your help is needed!

Shadsy · Post by **Shadsy** » Wed Sep 14, 2016 2:58 am

Hi there! I was an OHR community member many, many, many years ago. In my current life, I'm loosely, tangentially involved in game history/game preservation circles.

I've recently decided to undertake a large-scale project to preserve, as much as possible, the pre-FreeBasic games created by the OHR community. The end goal is to assemble every DOS-compatible game available, include as much ancillary detail about their contents as is feasible, and allow them to be played via the web through the Internet Archive's online emulation software, The Emularity.

This is going to take a while, but I'm going to get it done eventually. Nothing will probably be public from this project for months, but my goal is to create the most comprehensive, archival-quality collection of OHR history on the web.

I've been combing through a ton of sources, including the Castle Paradox archives, the O:OHR archives, Bahamut RPG Community, Rinku's OHR Hits collection, and other stray sources of links where I find them. (The OHR wiki has been particularly helpful for dealing with some of the 48 Hour Contests, and the Hamster Republic archive has been invaluable for older versions of the OHRRPGCE software and tools.)

But there are some holes where I'll need your help. There are people here who probably have tons of now-unavailable games stockpiled somewhere, and if so, now is your chance to lend a hand.

Currently, I'm attempting to track down the original version of Final Fantasy H, pre-2007 update, as well the "ingredients" file that was used for the Iron Chef-themed 48 Hour Contest from 2003. I realize these are both long shots, but I want to reach out to the community here because I figure y'all if anyone will have them.

Please help if you can on those two. I'll probably drop back in periodically as I find things that aren't available for download. Here's hoping something worthwhile comes out of this project!

Post by **TMC** » Wed Sep 14, 2016 5:33 am

Ugh, retyping my post because my computer crashed...

Would you happen to be the person formerly known as Shadowman?

I'm also very interested in preservation of the OHR's history, and was even considering organising uploading OHR games to the Emularity myself.

I'd like to create a central list of all known OHR games which links to downloads, descriptions, websites and reviews available on other sites, as well as creating backups of all of those.
The idea of creating a new game list for that sort of thing has been discussed a few times here.

(Maybe it could even work as an archival service in the same way as archive.org:
allow people to upload any games they possess, whether they're their games
or not, and I'll store them and try to work out whether I can publically host them.
What's archive.org's model, just host everything and take it down once someone complains?
But the game could be added to the public database even if it's not downloadable.)

Seven years ago
there was a site called The Hamster Wheel, which collected a huge amount of OHR information,
including a gamelist with a lot of hand-curated information on 1015 games, and collections of articles, contests, and a collection of original reviews. Sadly, the website went down and everything on it except for the reviews were lost, and archive.org only archived the indices of games, contests, etc, but none of the individual pages! :(
More recently, MBS relaunched THW as a much less ambitious site focusing on reviews. He hasn't updated it in a year, though.

I wrote a tool called "rpgbatch" to scan the public OHR gamelists, including CP, Op:OHR, SS, Bahamut, Rinku's OHRHITS, and a small additional collection from various source (but I haven't yet mirrored and included the hamsterrepublic.com gamelist), and analyse the .rpg files. For engine development it's useful to find games which contain corrupt data or depend on bugs or obscure features or script commands. You can see the game list here: http://tmc.castleparadox.com/ohr/gamelist/scangames.txt

Code: Select all

Run 1&#58; Finished in 205.12s
Scanned 2044 zips &#40;18 bad, 2 unsupported&#41;
Found 2049 RPGs, 9 corrupt, &#40;1606 unique, totalling 7402.3 MB&#41;

This output only shows the origin, filename, long name, and description. The table at the bottom also shows RPG file format version and last modified date. Exactly identical files are filtered out, but there are a lot of different versions of the same game.

You can see that there are four copies of FFH found. Two identical copies of FFHNORM.RPG on SS and CP (37.zip) from 2007, and FFHARD.RPG from 2003 (133.zip), and Ffh.RPG from 2002 on Op:OHR (Final%20Fantasy%20H.zip). I didn't see anything that looked like the original Chef ingredients file.

So the idea is to base the one-stop-shop gamelist on automated crawls like this, also crawl review lists, contest lists, description pages, and then allow adding hand-entry of more information. I would include games with no known download, pulling game names from sources like Charbile's biglist.

I think we should team up! If you find any games missing from this list, or websites hosting individual OHR games, send them my way.

Here are some more leads for you:

-MBS asked for help finding games for THW and some people responded, especially Pepsi Ranger. I'd like to get copies of all those games off him myself. I was thinking of creating a version of my utility to scan his games and report what he has.
-A filelisting of Op:OHR. A small handful of zip files aren't on the gamelist (I think they were linked to from articles)
-Rinku claims to have a copy of all games ever

Shadsy · Post by **Shadsy** » Wed Sep 14, 2016 2:17 pm

TMC wrote:Would you happen to be the person formerly known as Shadowman?

It sounds like we've got some common goals here, and it'd be great to pool resources on this! To outline my current scope and what I'm trying to collect...

For sanity, I've limited the project to just games with downloads available, not any of the ancillary things like reviews or music collections. That other stuff is important to collect eventually too, but the playable games are the priority and main interest.

I also put a date cap of September 21, 2007, since that was the release of the first non-DOS version of the engine.

For compactness and organization, I'm renaming all the zip files/storage directories/CP-provided screens of the games to the names of the .rpg files (so Final Fantasy H Hard Type would be in FFHARD.zip, games/FFHARD, and with screens/FFHARD.gif). I'm also removing extra copies of game.exe and similar programs/runtimes for the sake of file size, but I'm otherwise including all the readmes, concept art, directory structure, etc. that comes with them.

In cases where there's multiple RPGs in one zip, I'm naming the folder after the overall collection name (like "2006 Collaboration: Martin's Tale") but still listing each .rpg file separately.

The goal is to upload everything to the Internet Archive once the work is complete (I'm in touch with people in that community who have volunteered to help wrangle everything to make a batch upload possible). I definitely don't want to depend on other external servers for reliability. But even though the Internet Archive is terrific for hosting and browsing, it can be a little difficult to search through specific tags/metadata, so I'm planning on making all the information available elsewhere in a more sortable format, drawing the actual files/content from the Internet Archive uploads.

Everything is being stored in an Access database that I can export to MySQL or a CSV file. It also includes information about all the engine versions and additional tools (UNLUMP, RELUMP, PC Piano, etc.). I'm at work right now and can't reference it exactly, but from what I recall, the information I'm gathering is...

Name, as listed in the collection/website
Author
Long name
About line
Description if provided (may go back and add them later to ones missing descriptions)
Date game was completed
Date published if different/significant/not an artifact of when a collection was put online
Most compatible engine (closest new version when there's gaps in engine availability, closest old version when there's not)
Genre (RPG; non-RPG; or an experiment, like a plotscripting test or a sprite library)
Overall tone (Silly, like Arfenhouse; Slightly Silly, like Wandering Hamster; Serious, like Sword of Jade: Parallel Dreams; or Adult, like something puerile and dumb that doesn't really fit in those buckets)
.RPG Filename
Directory of the RPG file (when it differs or is kept multiple directories down)
Filesize of the .RPG file in kb (although not 1-to-1, this is a good indicator of the size of a QB game. I figure this might be something that can be autogenerated later, but just in case I'm jotting it down manually too)
Screenshot filename (if available; may create missing ones later, as with description)
Readme contents (for the sake of searchability. Stored as rich text)
Source (can be multiple if they have different incomplete information)
Collection (for series, 48 Hour Contests, etc. Can be multiple)
Uses QuickBasic features (for post-serendipity games that play in DOS but may have music issues)
Additional notes (like whether there are supplemental materials with the game; engine compatibility quirks, since some old games saved under a new engine only use old features; or other oddities, like one game that linked to a LiveJournal that's no longer online)

You're much more code-savvy than I am (I am but a librarian!), so if there's any of that you can automatically pull, particularly downloading the game/screens and renaming them based on the .rpg contents, that would be an enormous help. It seems like rpgbatch might already be able to do a lot of that! (Long name/about line/size of the rpg file too?)

That said, I'm still going through and doing manual curation, because I think that's really important for the sake of discovery. In addition to all the stuff I'm adding, I'm also keeping my own list of notable/interesting titles to highlight when all is said and done.

I'm happy to share the Access database and what specific fields I'm using to see if there's a way for the tools you made to automatically download/fill that stuff in, keeping in mind the date limits. If all this sounds good, I'm excited to work with you to improve the quality of the collection and take care of a whole bunch of work in one fell swoop.

We can talk specifics/technicals/etc. in PMs or emails if it keeps the boards cleaner.

The Wobbler · Post by **The Wobbler** » Wed Sep 14, 2016 2:23 pm

I guess I'll be the first jerk to say I really, really don't want my games archived anywhere I don't have control over, especially since one of them I removed from here in order to sell commercially. I don't want obsolete copies floating around nor do I want all of my one-note jokes that haven't been available in a decade showing up alongside my actual content.

RMZ · Post by **RMZ** » Wed Sep 14, 2016 2:30 pm

Yeah, I can agree. Basically all of my games not on my own website, I don't want people downloading. I still have yet to get a hold of anybody who can help me on Castle Paradox to remove my games there, which, if anybody can tell me who to talk to that'd be great since I can't get Inferior Minion to respond to my emails. I don't want to worry about my horribly offensive games getting out and compromising what I want to do now.

Post by **Bob the Hamster** » Wed Sep 14, 2016 6:02 pm

RedMaverickZero wrote:...I still have yet to get a hold of anybody who can help me on Castle Paradox to remove my games there...

If I remember correctly, you can upload a .zip file with the same name as the old .zip, and it should overwrite the old file.

Hopefully that is better than nothing.

As for Final Fantasy H, I have an old copy, but I don't know the file-date on it (it has the date of my last hard drive recovery). The filesize is 12900885 bytes, and the shasum is:

Code: Select all

9430c6943b62a54da5646acf9ad3d7154ae6944c  FFHNORM.RPG

I'm not sure if that is the same as whatever you already have, but if it is different, let me know, and I will send you a copy.

Shadsy · Post by **Shadsy** » Wed Sep 14, 2016 7:51 pm

Thank you! I'm at work currently, but later I'll see how it matches up with what TMC found.

---

So, I hadn't considered concerns about community members not wanting to make some of their old games available, which was a mistake. It's a major archiving ethical issue, so I want to address it head-on.

I think it's in the public interest to keep as much around as possible, even (and especially) out-of-date games and the embarrassing joke stuff, for the sake of documenting a relatively unknown slice of gaming history. Although games like Thanksgiving Quest or AXLE RPG (to bring up a direct example of a game you made, RMZ) are poor and probably outside good taste in some places, I think they speak to the attitudes and norms of online gaming communities in 2003-2004 better than any editorial that could be written. So I would hope those sort of games could be included with the understanding that the goal of the collection is historical.

That said, I understand and am sensitive to requests not to include things, either for commercial/intellectual property reasons or because something might reflect poorly. God knows I have an old internet trail I've spent time cleaning up. I do not want to proceed forward with this project without the community being comfortable with it and buying into it.

I've asked around in archiving circles, and there are a few ideas that I think would be appropriate.

I want to make the ability to opt out of public hosting for games as clear as possible – both now pre-emptively and later once the games are hosted. Since it's unfeasible or even impossible to contact the author of every game, the steps for requesting removal would need to be visible. (I would still hope to retain copies of things non-publicly for the sake of preservation 50 years from now.) I'm no longer convinced the Internet Archive is the right host for the project since that would abdicate control of the takedown request process and non-public content, though it would still be important to use their Emularity software to play the games on the web.

For these reasons, I think it might be appropriate to turn the reins of this project over to TMC. You're more embedded in the community, would likely be able to provide near-to-long-term hosting and asset management under your own control, and could respond to concerns like the ones Wobbler and RMZ raised more quickly.

I don't want to burden you by suddenly showing up and giving you a digital free puppy (you get to feed it and clean up after it!), because you probably have enough on your plate. But since it sounds like something that you already have a strong interest in and have already been developing tools for, I hope you'd be interested in picking up what I have done so far.

It's mostly groundwork right now, and I have about 10% of the Castle Paradox archives account for, so I would be able to hand over what's completed and help you hit the ground running fairly easily. I'm eager and glad to help with Emularity configuration, making sure the database translates well to a discoverable web format, and archival best practices/etc. questions, but based on what it looks like you've done so far, I think you're better equipped to deal with the project on a day-to-day basis.

(And you can retrieve copies of games and their metadata more efficiently than I can by hand! It might even be possible to complete this before my own expectation of "mid-April.")

---

So the short version: I'm hoping TMC could take control of the project to make it more responsive to the community's interests, and if you would like to opt out of including some or all of your games, those concerns are seriously noted. (Though please consider allowing games to stay in for historical reasons.)

Gizmog · Post by **Gizmog** » Wed Sep 14, 2016 9:04 pm

There's no history to be preserved in most of it, it's like archiving every kindergarten doodle anyone ever did. I mean for christs sakes, an ingredient file for a one-off contest? The lost works of Plato it ain't. Final Fantasy H makes sense, I guess? Arfenhouse definitely makes sense. But most of that stuff is junk and should be left alone. There's enough shovelware/vaporware without getting historical stuff mixed in there too.

The Wobbler · Post by **The Wobbler** » Wed Sep 14, 2016 10:41 pm

Giz's post is pretty much where I fall on this. A database of important/good games would be subjective, but more useful and easier to get author consent than just throwing up everything ever. Like, there's no historical value to preserving a two minute joke I threw up in response to a forum post. It's not a game.

Then again I don't like the Internet Archive project at all. I know this is a philosophical difference, but I think it's a pretty gross privacy violation. We should be free to say dumb slime and look stupid as kids without it existing forever.

Shadsy · Post by **Shadsy** » Thu Sep 15, 2016 12:38 am

I think I misjudged the community's interest based on the extent of the collections currently available online. I'm surprised by the hostile reception.

I understand archival consent concerns, and that's a tricky ethical issue that I'm eager to work through. But if there's such a vocal, ideological opposition to the concept and, apparently, an active effort to remove content from those other collections, going forward with the community's involvement isn't possible. I would want a large-scale OHR archive to be a celebratory snapshot of an exciting period in online gaming culture, weird parts and all, and that isn't going to work if it draws the community's ire.

Consider the project discontinued. This is a net loss, but I am clearly not the right person to attempt to work with the community on this.

Gizmog · Post by **Gizmog** » Thu Sep 15, 2016 12:50 am

The Wobbler wrote:...I think it's a pretty gross privacy violation. We should be free to say dumb slime and look stupid as kids without it existing forever.

Exactly! It's a super-noble sentiment, and I support the efforts to recover lost media because there are some things out there that people worked hard on and deserve to be seen. Wisdom that shouldn't be forgotten, that kind of thing. But remembering *EVERYTHING* on the internet really only leads itself into the ultimate realm of cyber-bullying.

People have a right to be forgotten. Seriously, every time my mom brings up some embarassing story of some horrible thing I did as a kid, I realize this as a universal truth. Personally, I take a great deal of solace in the fact that one day my existence will have had no notable effect on anything or anyone and really don't need some internet clods doing away with that.

And look at it this way: If the NSA was doing this, people would be rioting in the streets.

EDIT:

Shadsy wrote: I would want a large-scale OHR archive to be a celebratory snapshot of an exciting period in online gaming culture, weird parts and all, and that isn't going to work if it draws the community's ire.

Consider the project discontinued. This is a net loss, but I am clearly not the right person to attempt to work with the community on this.

Okay, that makes me really sad. On a more encouraging note, Meatballsub has been engaged in an archival effort of his own over here at Hamster Wheel. Archiving reviews seems like a pretty good way to do it, "There was a game, it was stupid and embarassing" is way better than having your Presidential Campaign derailed because someone posts a game you made in 3rd grade. (PS: If Trump or Clinton made any OHR Games, please archive them forever. I don't know which one of them is worse.)

Foxley · Post by **Foxley** » Thu Sep 15, 2016 12:59 am

Man what? I'd personally be thrilled to see old, old embarrassingly-bad-yet-charming OHR games in an easily playable collection, with the caveat that attempts would be made to inform the original author and ask their permission, especially if they put their RL name all over it. I mean, making a game with awful middle schooler humor and having it come back to haunt you now that you're working in a professional career in your 30s could potentially do some damage.

If you put something up for unrestricted public perusal at any point, even if it was nearly 2 decades ago, then I'm sorry but it's kind of out of your hands. At least if you ask someone to take it down in this case, they will. Informing people and asking consent is just the civil thing to do when dealing with redistributing orphaned software.

I think at least some quality control would need to be exercised to improve signal to noise ratio, like if it's someone's "my frist rpg demo" and it has yellow squares on a green background and almost no content, that kind of thing should be dismissed, of course.

Don't let literally two naysayers convince you to abandon a potentially rad project. I for one would be interested.

The Wobbler · Post by **The Wobbler** » Thu Sep 15, 2016 1:19 am

Foxley wrote:with the caveat that attempts would be made to inform the original author and ask their permission

This is really the thing that matters the most to me. If lots of people are on board and want their games archived, that's great! I know it's next to impossible in some cases. I still think it's the fair thing to do.

Post by **TMC** » Thu Sep 15, 2016 2:01 pm

I'm still keen to put this together. These reactions don't surprise or dissuade me; I know that a lot of (most?) people don't want to be associated with the games they made when they were 12 -- I don't either! I was 14 when I registered on CP, and it's not pretty. I removed a game or two I had on CP (though I still have plenty of embarassing ones there), though I do actually regret not even having a personal copy remaining. I know many people have not reuploaded their older games to SS. Even Mogri has something like a dozen half-decent games on CP that aren't on SS. I certainly would give people control over their own content in the game listing, which is why, as you suggested, I will host it myself rather than put it elsewhere like archive.org. (That doesn't mean that we can't put some good DOS OHR games that people are proud of on the Emularity.)

I think it would be nice to censor games to the minimum degree necessary. For example maybe you could opt to list your games anonymously so that people can't stumble upon it by searching for your name/handle or your other games on Google (that's clearly not sufficient for e.g. the Mr. Triangle games). In addition to the options of not hosting an independent download (if I have a copy not available elsewhere) or making the listing completely hidden. Also, it could be possible to tell web search engines not to index specific game entries.

If games are already listed on other public sites, then I feel there's no reason I shouldn't default to making all game information public; you should get your game taken down from SS/CP/HR/the OHR wiki if you're not happy with it being public. I realise you can't easily get something removed from the likes of Bahumut or HamsterSpeak though (would you delete a review someone didn't like?)

The Wobbler wrote:A database of important/good games would be subjective, but more useful and easier to get author consent than just throwing up everything ever

Actually I don't agree. My goal is mainly to provide a searchable list of games, reviews and resources that are already public. People do often ask about old copies of games they can't track down, or may want to do some research, e.g. for a retrospective, so it's more useful as a resource to list everything than manually guess what is interesting. It's also actually less work to put everything up rather than curate.

I would however also like to provide separate lists of the better games. It would be great to collect all the reviews, top 30 lists, game rankings, as well as retrospectives and previews. For example, a lot of contest games get voted on and review in a thread somewhere, and those reviews almost never make it onto a review list. I can also provide (optional) filters for no-substance joke games: I can check the number of textboxes, maps, battles, etc. a game has. That's really what the appeal of the site would be.

I hadn't really thought about making the games playable on the web in an emulator too; that would be neat.

RedMaverickZero wrote:I still have yet to get a hold of anybody who can help me on Castle Paradox to remove my games there

IM never responded to my last email either. As James said, the way to remove a game from CP is to upload a blank zip, and optionally change the description/screenshot/name. If you want to talk to an admin who can do more, try PMing or emailing Cube, he's an admin and had a lot of responsibility. I'm also an admin on CP, but I was made one only to delete spam, not games, so I've already refused previously to do so.

The Wobbler · Post by **The Wobbler** » Thu Sep 15, 2016 3:18 pm

I think listing links to places where you can currently, publicly download games is fine. I don't like rehosting people's content, but providing a database of direct links makes plenty of sense.