{"id":13993958,"url":"https://github.com/mmiszczyk/LondonRising","last_synced_at":"2025-07-22T18:32:46.950Z","repository":{"id":139370364,"uuid":"117289234","full_name":"mmiszczyk/LondonRising","owner":"mmiszczyk","description":"Reversing Fallen London","archived":false,"fork":false,"pushed_at":"2018-04-27T19:49:34.000Z","size":5547,"stargazers_count":24,"open_issues_count":0,"forks_count":5,"subscribers_count":4,"default_branch":"master","last_synced_at":"2024-11-29T15:50:41.681Z","etag":null,"topics":["narrative-network-analysis","reverse-engineering","video-game"],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mmiszczyk.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2018-01-12T21:23:07.000Z","updated_at":"2024-09-24T02:53:17.000Z","dependencies_parsed_at":"2024-01-18T04:51:22.393Z","dependency_job_id":"c2938203-caa4-4c02-9feb-7db4c27befaf","html_url":"https://github.com/mmiszczyk/LondonRising","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/mmiszczyk/LondonRising","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mmiszczyk%2FLondonRising","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mmiszczyk%2FLondonRising/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mmiszczyk%2FLondonRising/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mmiszczyk%2FLondonRising/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mmiszczyk","download_url":"https://codeload.github.com/mmiszczyk/LondonRising/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mmiszczyk%2FLondonRising/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":266552498,"owners_count":23947174,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-07-22T02:00:09.085Z","response_time":66,"last_error":null,"robots_txt_status":null,"robots_txt_updated_at":null,"robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["narrative-network-analysis","reverse-engineering","video-game"],"created_at":"2024-08-09T14:02:38.641Z","updated_at":"2025-07-22T18:32:46.942Z","avatar_url":"https://github.com/mmiszczyk.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"```\n7MMF'                                 `7MM                        \n  MM                                     MM                        \n  MM         ,pW\"Wq.  `7MMpMMMb.    ,M\"\"bMM   ,pW\"Wq.  `7MMpMMMb.  \n  MM        6W'   `Wb   MM    MM  ,AP    MM  6W'   `Wb   MM    MM  \n  MM      , 8M     M8   MM    MM  8MI    MM  8M     M8   MM    MM  \n  MM     ,M YA.   ,A9   MM    MM  `Mb    MM  YA.   ,A9   MM    MM  \n.JMMmmmmMMM  `Ybmd9'  .JMML  JMML. `Wbmd\"MML. `Ybmd9'  .JMML  JMML.\n\n             ----------------------\n             |\u0026\u0026\u0026\u0026\u0026\u0026\u0026\u0026\u0026\u0026\u0026\u0026\u0026\u0026\u0026\u0026\u0026\u0026\u0026\u0026|\n             |\u0026\u0026\u0026\u0026\u0026\u0026\u0026\u0026\u0026\u0026\u0026\u0026/.     *|\n             |@@@\u0026\u0026\u0026\u0026\u0026\u0026\u0026\u0026\u0026\u0026@@@\u0026*.%|\n             |@@@@@@@@@@@@@@@@@@#.|\n             |@@@@@@@@@@@@@@@@#.. |\n             |@@@@@@@@@@@@@@@@#.. |\n             |\u0026@@@@@@@@@@@@@@@@@#.|\n       /----\u0026(\u0026@@@@@@@@@@@@@@@@@,@\u0026----\\  \n      @\u0026\u0026\u0026\u0026\u0026\u0026\u0026\u0026\u0026\u0026\u0026\u0026 \u0026\u0026\u0026\u0026\u0026\u0026\u0026\u0026 \u0026\u0026\u0026\u0026\u0026\u0026\u0026\u0026\u0026\u0026\u0026@                         Fallen London\n     (%\u0026\u0026\u0026\u0026\u0026\u0026\u0026\u0026\u0026\u0026\u0026 ' \u0026\u0026\u0026\u0026\u0026\u0026 ' \u0026\u0026\u0026\u0026\u0026\u0026\u0026\u0026\u0026\u0026\u0026)          Reverse Engineering Project\n      @\u0026\u0026\u0026\u0026\u0026\u0026\u0026\u0026\u0026\u0026\u0026   \u0026\u0026\u0026\u0026\u0026\u0026   \u0026\u0026\u0026\u0026\u0026\u0026\u0026\u0026\u0026\u0026@\n       \\-------------------------------/  \n               \\                */        \n                \\  / v v v v.\\  /\n                 \\/           \\/\n                 \n              ,,            ,,                  \n`7MM\\\"\"\"Mq.    db            db                                     \n  MM   `MM.                                                        \n  MM   ,M9  `7MM  ,pP\"Ybd `7MM  `7MMpMMMb.   .P\"Ybmmm              \n  MMmmdM9     MM  8I   `\"   MM    MM    MM  :MI  I8                \n  MM  YM.     MM  `YMMMa.   MM    MM    MM   WmmmP\"                \n  MM   `Mb.   MM  L.   I8   MM    MM    MM  8M                     \n.JMML. .JMM..JMML.M9mmmP' .JMML..JMML  JMML. YMMMMMb               \n                                            6'     dP              \n                                            Ybmmmd'     \n```\n\n# Introduction\n\nCan it be true? All the world's knowledge, all the secrets - but no\nsacrifice required? Can everything about your existence and all your\npossible existences be deciphered from the inhuman languages and their\nWords-Behind-Worlds? Is that dusty old tome the real Reverse Engineer's\nJournal - and if it is, is it as powerful as they claimed? And perhaps\nmost importantly: what is the real price of such power?\n\n+ **Shadowy** is increasing...\n+ **Watchful** is increasing...\n+ You've gained 1x **Reverse Engineer's Journal**\n+ You've gained ∞ **Fate**!\n+ A twist in your tale! You are now **A Scholar of Words-Behind-Worlds**\n\n# Actual Introduction\n\nFallen London is a text-based browser MMO. It is set in a very\ninteresting world and its writing might be among the best in gaming.\nUnfortunately, it is a browser MMO so all that good stuff is locked\nbehind both microtransaction paywalls and endless hours of grinding.\nFor a more detailed analysis of the game (and other titles set in\nthe same world), feel free to read\n[an article](https://hg101.kontek.net/fallenlondon/fallenlondon.htm)\nthat I've written for Hardcore Gaming 101 (#TODO: link to a new HG101\nsite when they migrate that article).\n\nFortunately for us, we don't have to play the boring game of Fallen\nLondon to learn the interesting story of Fallen London. Instead, we can\nplay a much more interesting game of reverse engineering.\n\nWe have talked a bit about this project with Failbetter Games and came\nto an agreement that it would be better not to reveal some of the more\ntechnical details about our findings as we don't want to enable cheating\nand piracy of Fate-locked (i.e. paid) content. If you want an in-universe\nexplanation, let's say that you were unable to acquire a copy of\n**Reverse Engineer's Journal** before Ministry of Public Decency found it.\n\n# Graphing the Neath\n\n## From JSON-in-SQLite to NetworkX to GEXF\n\nThe decrypted database contains more or less the entire design of\nthe game. There's just one problem with that: we have no convenient way\nof exploring it. Sure, we have a readable SQLite - but the game data\nisn't really described in a relational way so it's not very useful. You\nsee, there are no foreign keys here. It's JSONs with ID fields that\nrefer to ID fields of other JSONs when they want to make a reference.\nIt's like a second database inside the first one.\n\nSpeaking of references, there are a lot of those. Most of the in-game\nobjects contain, are contained by or in other ways connect to other\nobjects. One-to-one, one-to-many and many-to-many relations all exist in\nhere - and to understand what's going on, we need a way to easily follow\nthe links. Pulling the 'internal database', figuring out its structure\nand turning into actual, well-designed relational DB might be a good\nidea if we wanted to make our own version of the game and cared about\nperformance but such a structure would be quite tedious to explore\ninteractively as we attempt to untangle Fallen London's narrative.\n\nGiven that Fallen London can be thought of as a hypertext game,\na natural representation might be **H**yper**t**ext **M**arkup\n**L**anguage. I of course reject this perfectly sane solution because\nI'd like something easier to manipulate in an automated fashion\n(the dataset is big so we want to reduce the tedium as much as\npossible).\n\nBecause Fallen London can be thought of as a hypertext, a hypertext can\nbe thought of as a graph and JSONs in the database can be thought of as\na second database, my first thought is to use a graph database like\nNeo4J. Unfortunately, there seems to be no Neo4J (or similar) equivalent\nto SQLite that can be stored in a single file. They all need to be set\nup as a server, which I feel is just not worth the trouble as it more or\nless guarantees that nobody reading this will want to waste time on\nthat - and what good is a reversing writeup if the readers can't be\nbothered to follow it?\n\nIn what in retrospect might not be such a good decision, I decide to\ndrop the 'database' part of 'graph database' and just store everything\nas a graph that can be then accessed by graph exploration and network\nanalysis software. I use the NetworkX library to represent game data as\nnodes and edges, write the fl_types module to tell it how to convert\ndifferent types of game objects and then dump the output to GEXF format\n(it's based on XML because of course it is).\n\n## Working with Gephi: the big graphs\n\nI do a bit of research about open-source network analysis tools and go\nfor Gephi. It has a Python scripting plugin, it's\n[aesthetic as hell](https://player.vimeo.com/video/9726202) and it has\nalready been used in some interesting narrative network analysis\nprojects - e.g. Andrew Beveridge's\n[Network of Thrones](https://www.macalester.edu/~abeverid/thrones.html)\nwhich used it to analyze the relations between the characters in A Song\nOf Ice and Fire novel series. What could possibly go wrong?\n\nWell, for the starters, Gephi just can't read my GEXF. It claims that\nsome node or edge ID is null, even though it isn't. Googling reveals\nthis to be\n[a known problem for large files](https://github.com/gephi/gephi/issues/1841).\nBecause as of writing it haven't yet been fixed, I modify my scripts\na bit to ignore some objects and to treat some references as if they\nwere attributes, thus limiting the number of nodes and edges. This\nworks, and we're given this beautiful representation of game data:\n\n![](images/graphs/unreadable-graph.png)\n\nI play around with colors a little bit (I color the nodes based on\npartition-\u003etype, with a custom palette generated from\n'Fancy (dark background)' preset), set the sizes of the nodes based on\ntheir indegree, make the background black and then try to find a decent\nlayout. Unfortunately, most of the layouts either result in a huge mess\nor take forever to generate - both of those problems arising due to\nthe size of my graph. Really, the only one that is even remotely useful\nhere is 'Open Ord'.\n\nI also make labels for each node based on the 'Name' attribute (if you\nhave Gephi scripting plugin installed,\n[there's a script for that](gephi_scripts/generate_labels.py))\nand scale their size based on indegree. I display them in\na nice Palatino font (I think Failbetter uses Adobe's Trajan Pro 3 but\nI don't have money for it). Unfortunately, you can't really see much of\nthe text in this image - but if you look hard enough, you'll see\nthe words 'Key of Dreams'. That's a quality which\n[only the developer accounts have](http://community.failbettergames.com/topic21866-what-is-the-key-of-dreams.aspx),\nand setting it as a required quality for something blocks it for normal\nusers. This means that there's a lot of such locked content in the game\n(other quality used for locking is 'Impossible'; it's also required for\na lot of things).\n\n![](images/graphs/big-graph.png)\n\nAfter trimming the graph a little bit (I removed things which don't have\na 'type' attribute - as they're result of some kind of a bug - with\nFilters-\u003eLibrary-\u003eAttributes-\u003eNon-null-\u003etype (Attribute); I also removed\nthings that are not connected to the main graph with\nFilters-\u003eLibrary-\u003eTopology-\u003eGiant Component). Then, I used the Dual\nCircle layout, made 8 upper order nodes based on the indegree and placed\nthem outside of the main circle. This gave us this cool-looking star:\n\n![](images/graphs/8-point-star.png)\n\nAs you see, the locations ('Somewhere else', 'Veilgarden',\n'your Lodgings', 'The University') are connected to many other things.\nIn subsequent analysis, this made isolating graphs for specific\nquestlines in an automated way difficult so I modified a script to\nignore them so that most edges would connect events and qualities.\n\n## Using modularity to find related content\n\nCalculating modularity is one of the methods of (non-overlapping)\ncommunity detection. Nodes within those communities will be densely\nconnected with each other but sparsely connected to the rest of\nthe graph. As a result, a graph will be effectively divided into\nsubgraphs.\n\nModularity is useful to us: a group of interconnected events, qualities,\nbranches etc. is highly indicative of a discrete storyline.\nUnfortunately, this isn't so simple in practice: the algorithm tends to\ncreate both large communities that would be better off split up into\nsmaller sections and micro-communities of 2 or 3 nodes that don't tell\nus much. Finding desired results will require a lot of manual tweaking.\n\nTo run the algorithm, use Statistics-\u003eNetwork Overview-\u003eModularity. You\ncan then filter the graph by detected community with\nFilters-\u003eLibrary-\u003eAttributes-\u003ePartition-\u003eModularity Class.\n\nWith the modularity algorithm, we can find content related to our\nplayable character's main stats. This is nice, but it also reveals how\nthe presence of such 'popular' vertices distorts our results a bit:\n'those things are all related to your persuasion skill' is a very broad\nstrokes connection when this skill is connected to (more or less) one\nfourth of the game's content.\n\n![](images/graphs/persuasive.png)\n\n![](images/graphs/shadowy.png)\n\n![](images/graphs/dangerous.png)\n\n![](images/graphs/watchful.png)\n\nBut, as I said, with a bit of tweaking, you can find what you're looking\nfor. Here's a graph representing the game's infamously difficult quest\ncalled 'Seeking Mr Eaten's Name'. We're digging into the game's secrets\nhere: this is basically Fallen London's endgame, and very few players\nhave finished it.\n\n![](images/graphs/smen.png)\n\nKeep in mind that the modularity algorithm is just a heuristic: while\nmost of the things you see here are related to SMEN, I don't think that,\nfor example, 'A Game of Chess' (one of the game's sort-of-PVP\ncomponents) has anything to do with it.\n\n## Ego networks/Neighborhood networks\n\nEgo network consist of a vertice, all the vertices connected directly\nto it and all the edges between them. This is obviously a good way to\nshow things around a central node. You can find the node's ego network\nin Gephi with Filters-\u003eLibrary-\u003eTopology-\u003eEgo Network and inputting its\nID, and if you want to make an ego network for a group of nodes that\nyou've filtered then drag Filters-\u003eLibrary-\u003eTopology-\u003eNeighborhood\nNetwork into its subfilters.\n\nIn game terms, this is useful for finding stuff related to a specific\nquality. For example, we can get all the choices that aren't accessible\nto the players by making ego networks for 'Impossible!' and 'Key\nof Dreams' qualities.\n\n![](images/graphs/inaccessible.png)\n\n## Ambitions: the final boss of Fallen London RE graphs\n\nIt is said that a well designed final boss fight is like a final exam,\ntesting everything you've learned while playing the game. In the game of\nreverse engineering Fallen London into cool-looking graphs, we're going\nto make the ultimate cool-looking graph with our knowledge of Gephi\nlayouts, filters, modularity and ego networks. Ladies and gentlemen,\nit's time for Ambitions!\n\nIn Fallen London, Ambitions can be though of as 'main quests'. They last\na really long time, finishing them requires grinding related stats to\nquite a high level (or it would if they were actually possible to\ncomplete; as of writing, they're still unfinished), they can have\ndiscrete subquests and you can only have one active at the same time.\n\nThere are four ambitions in the game: Nemesis, Heart's Desire, Light\nFingers and Bag a Legend. Due to their size, they connect to a lot of\nthings, but there's one thing that makes working with them easier:\nqualities related to your main quest progress have titles like\n'Ambition: [ambition name]', or\n'Ambition: [ambition name] - [subtitle]'. This means that we can match\nthem with simple filters: Filters-\u003eLibrary-\u003eAttributes-\u003eEquals-\u003eName\nwith values defined as 'Ambition: Nemesis.\\*', 'Ambition: Heart.\\*',\n'Ambition: Light.\\*' and 'Ambition: Bag.\\*' and the box that turns\nvalue to regex ticked. Now we can drop the Neighborhood Network into our\nsubfilter and we're good to go.\n\nWell, we would be, if I didn't lie about one thing. You know, there's\nactually five ambitions. The last one is another one of the game's\nsecrets. It's called 'Ambition: Enigma' and it works in a completely\ndifferent way from other ambitions. It's very short, it's hidden behind\nsome Invisible Cities references (or rather 'Invisible Cities references\nthat are more explicit than the usual Invisible Cities references in\nFallen London') and it's less about the world of the game and more about\nthe game itself. So we need just one more ego network, but this is\ntrivial to create.\n\nNow that we have a subgraph of all the ambition-related content, it's\ntime to make it look cool. First of all, we create filters for\nqualities that track the progress of main quests. This works just like\nthe filters in the above paragraph, but without Neighborhood Networks\naround them. We also make a filter for things that are not qualities\nthat track progress directly: we need Filters-\u003eLibrary-\u003eOperator-\u003eNOT,\nwith Filters-\u003eLibrary-\u003eOperator-\u003eUNION as its subfilter, and then we\nduplicate all the other filters and drop them as subfilters to that one.\n\nUnfortunately, this doesn't work too well for Enigma, as it's just two\nnodes called 'Ambition: Enigma' now. Fortunately, we can approach it\ndifferently. Because of its small size and the fact that it's not\nconnected to anything else, we can easily find a modularity which\nincludes the whole thing. We do it, and update the NOT filter\naccordingly.\n\nWhat follows is more art than science as I try to make the best-looking\ngraph I can. The four inner circles are four ambitions, the outer circle\nis parts of their ego networks that don't track the quest progress\ndirectly, and that spiral outside of the main graph is Enigma.\n\n![](images/graphs/ambitions.png)\n\n# Additional thoughts\n\n+ I found some things about Gephi really annoying: it couldn't read\nlarge files, it sometimes corrupted its workspace (which meant that I\nhad to start over because I couldn't open it); I don't want to be too\ncritical about it because I don't think that the output of my scripts\nis anything like the data that is usually fed to Gephi, but I want to\nwarn you: following my path will lead to a lot of annoyance\n+ While I don't want to be too critical about Gephi itself, I am going\nto be extremely critical about its scripting console plugin; a lot\nof the features described in\nthe [tutorial](https://github.com/gephi/gephi/wiki/Scripting-Plugin)\n(filter syntax, accessing graph node attributes like Python object\nattributes etc.) just don't seem to work\n+ When I found an interesting subgraph, I went into Data Explorer\nto read the descriptions - this worked but it wasn't very convenient;\nif I went the HTML generation route, reading would be convenient because\nI'd just use browser for it, but finding interesting content would be\na pain in the ass; a decent compromise would be using the scripting\nconsole to generate HTMLs from filtered subgraphs, but the problem is\nthat the scripting console sucks; I might do it if I'm bored though\n+ Maybe using a graph database would allow me to avoid some of\nthe annoyances I'm describing right now - I need to experiment with\nNeo4J\n+ I don't think anyone did something like that before - using graphs to\nreverse engineer a hypertext game (I might be wrong; feel free to send\nme examples of people doing that); despite the annoyances, I think it's\nreally interesting and I'm planning to do something like that in\nthe future; I'll probably try reversing *80 Days*, another British\nhypertext game with an alternate history Victorian setting\n+ One thing that was always on the back of my head when doing this:\nProlog; while I think that the exact things I was doing in this project\n(communities, ego networks, visualizations, interactive exploration) are\nbetter served by graph-specific tools, I do think that logic programming\nmight also be useful in reversing hypertext games - maybe for generating\n'walkthroughs' from game data (especially if desired outcome depends not\njust on going from one vertice to another but also on some conditions\nheld in variables); maybe I'll do something about logic programming\nfor hypertext RE\n\n# Authors\n\nLondon Rising was brought to you by\n[Maciej Miszczyk](https://github.com/mmiszczyk) and\n[Mateusz Lindenberg](https://github.com/Lindeed). Of course, it wouldn't\nbe possible without Alexis Kennedy and Failbetter Games.\n\nSpecial thanks to Adam Myers from Failbetter Games for talking to us\nabout things which can be brought to light and those that should\nremain secret.\n\n# License\n\n```\nThis program is free software; you can redistribute it and/or modify\nit under the terms of the GNU General Public License as published by\nthe Free Software Foundation; either version 2 of the License, or\n(at your option) any later version.\n\nThis program is distributed in the hope that it will be useful,\nbut WITHOUT ANY WARRANTY; without even the implied warranty of\nMERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the\nGNU General Public License for more details.\n\nYou should have received a copy of the GNU General Public License\nalong with this program; if not, write to the Free Software\nFoundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA\n```","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmmiszczyk%2FLondonRising","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmmiszczyk%2FLondonRising","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmmiszczyk%2FLondonRising/lists"}