{"id":20114070,"url":"https://github.com/davidb/forum-migration","last_synced_at":"2026-06-10T10:31:33.154Z","repository":{"id":23743090,"uuid":"27116958","full_name":"davidB/forum-migration","owner":"davidB","description":"Scripts for the great forum migration..","archived":false,"fork":false,"pushed_at":"2015-01-17T12:43:26.000Z","size":280,"stargazers_count":3,"open_issues_count":2,"forks_count":0,"subscribers_count":10,"default_branch":"master","last_synced_at":"2025-03-02T18:51:29.244Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Ruby","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/davidB.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2014-11-25T08:52:40.000Z","updated_at":"2019-08-25T12:52:51.000Z","dependencies_parsed_at":"2022-07-25T10:32:04.732Z","dependency_job_id":null,"html_url":"https://github.com/davidB/forum-migration","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/davidB/forum-migration","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/davidB%2Fforum-migration","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/davidB%2Fforum-migration/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/davidB%2Fforum-migration/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/davidB%2Fforum-migration/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/davidB","download_url":"https://codeload.github.com/davidB/forum-migration/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/davidB%2Fforum-migration/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34149132,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-10T02:00:07.152Z","response_time":89,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-13T18:27:59.393Z","updated_at":"2026-06-10T10:31:33.137Z","avatar_url":"https://github.com/davidB.png","language":"Ruby","funding_links":[],"categories":[],"sub_categories":[],"readme":"\nThis is my notes about migration from bbpress to discourse.\nWe used a customized script :\n* to includes likes (done by fake users)\n* generate files with mapping bbpress id / discourse id, to be able to update some data after main migration\n* generate Redirection for Discourse (alternative: to configure an url rewrite rule)\n* convert subscriptions into watching (we didn't run it at end)\n\n# SETUP Local Mysql DB to store bbpress data\n\n## Install Mysql and start it\n\nI installed mysql/mariadb on my desktop.\n```\n# Desktop Archlinux as root (see https://wiki.archlinux.org/index.php/MySQL)\npacman -S mariadb\nmysql_secure_installation\nsystemctl start mysqld\n```\n\nTo avoid the issue \"Cannot load from mysql.proc. The table is probably corrupted\", that can be raise by mysql2 + upsert\n\n```\nmysql_upgrade -uroot -p\n```\n\n## Setup bbpress database\n\nI create a working copy of the bbpress DB to migrate:\n\n```\nmysql -u root -p \u003c\u003c __EOF__\nCREATE USER 'bbpressUser'@'localhost' IDENTIFIED BY 'bbpressPwd';\nDROP DATABASE IF EXISTS bbpress;\nCREATE DATABASE bbpress;\nGRANT ALL PRIVILEGES ON bbpress.* TO 'bbpressUser'@'localhost' WITH GRANT OPTION;\n__EOF__\n\nmysql -u bbpressUser --password=bbpressPwd bbpress \u003c latestdump.sql\nmysql -u bbpressUser --password=bbpressPwd bbpress \u003c purge.sql\n```\n\n# SETUP Local Discourse\n\n## Install docker and start it\n\nsee you system doc, for archlinux:\n\n```\n# Desktop Archlinux as root\npacman -S docker\nsystemctl start docker\n```\n\n## Install discourse-docker\n\nInstall discourse on docker (see https://github.com/discourse/discourse/blob/master/docs/INSTALL-digital-ocean.md )\nbut don't call \"./launcher bootstrap app\" before reading the following note to avoid sending email to users during test/migration.\n\nI added discourse.localhost in my /etc/host\n```\n# Desktop Archlinux as root\necho \"127.0.0.1 discourse.localhost\" \u003e\u003e /etc/hosts\necho \"127.0.0.1 smtp.discourse.localhost\" \u003e\u003e /etc/hosts\n```\n\nI configured smtp with a invalid server to not send email.\nI will configure it right to activate admin, temporary later to avoid send email during migration test\nAs I use my gmail address, I'll use smtp server from google for initial activation of admin account.\n```\n# /var/discourse_docker/container/app.yaml:\n\n  DISCOURSE_DEVELOPER_EMAILS: 'me@gmail.com'\n  ##\n  ## TODO: The domain name this Discourse instance will respond to\n  DISCOURSE_HOSTNAME: 'discourse.localhost'\n\n  ##\n  ## TODO: The mailserver this Discourse instance will use\n  DISCOURSE_SMTP_ADDRESS: smtp.discourse.localhost  # (mandatory)\n  #DISCOURSE_SMTP_PORT: 25                        # (optional)\n  DISCOURSE_SMTP_PORT: 587                        # (optional)\n  DISCOURSE_SMTP_USER_NAME: me@gmail.com      # (optional)\n  DISCOURSE_SMTP_PASSWORD: *********             # (optional)\n```\n\nto prepare the migration, forward access of container port 3306 to host port 3306 so script in container can access my local mysql (on host):\nI replaced a line around 487 in /var/discourse_docker/launcher (to allow docker to access local mysql):\n\n    exec ssh -o StrictHostKeyChecking=no root@${split[0]} -p ${split[1]}\n\nby\n\n    exec ssh -R 3306:localhost:3306 -o StrictHostKeyChecking=no root@${split[0]} -p ${split[1]}\n\nIf you boostrap it before made the change, you can call \"./launcher rebuild app\" (will take few minutes ~10)\n```\ncd /var/discourse_docker\n./launcher boostrap app\n./launcher start app\n```\n\nTry access `open http://discourse.localhost/`\nregister an account with the your developer email (eg: me@gmail.com) You should not receive activation email. (see Commands below for an alternative way to activate account, without smtp server).\n\nOpen a shell into container\n```\n./launcher ssh app\ncd /var/www/discourse\n```\n\n\nTemporary enable smtp (you should be root inside the container)\n\n```\n# set the right smtp conf\nsed -i 's/smtp.discourse.localhost/smtp.gmail.com/' /var/www/discourse/config/discourse.conf\nsv restart unicorn  # restart discourse service inside the container\n```\n\n\nre-open http://discourse.localhost/, try to log-in and request resend of activation\nyou should receive it\nlog-in, open admin panel \u003e email, send a test email\nyou should receive it\n\n```\n# set the wrong smtp conf\nsed -i 's/smtp.gmail.com/smtp.discourse.localhost/' /var/www/discourse/config/discourse.conf\nsv restart unicorn  # restart discourse service inside the container\n```\n\nRe-open http://discourse.localhost/, log-in, open admin panel \u003e email, send a test email\nyou should NOT receive it, an error notification should nbe display on admin panel\n\n## Setup to speed migration\n\nBy default discourse download images on local, including remote image links, gavatar,... This step could take lot of time after the end of the migration script.\nI suggest you to **disable download remote images to local** for migration:\n[Admin/Settings/Files](http://discourse.localhost/admin/site_settings/category/files). After the migration, you can re-enable it.\n\n## Install stuff for migration from mysql\n\nInstalled stuff is available until you detroy / rebuild the container\n\n```\n# Container as root\n# install mysql2 ruby gem for migration script (take time)\napt-get install libmysqlclient-dev\n# bundle exec gem install mysql2\ncd /var/www/discourse\necho \"gem 'mysql2'\" \u003e\u003e Gemfile\necho \"gem 'upsert'\" \u003e\u003e Gemfile\nbundle install --no-deployment\n```\n\n## Install stuff to convert post_content\n\n```\n# Container as root\ncd /var/www/discourse/tmp\ngit clone https://github.com/nlalonde/ruby-bbcode-to-md.git\ncd ruby-bbcode-to-md\ngem build ruby-bbcode-to-md.gemspec\ngem install ruby-bbcode-to-md-0.0.13.gem\n```\n\n## Run ruby scripts\n\nCreate the script as script/import_scripts/bbpress_2.rb\nby default (security) I commented import_XXX instructions\n\nTo copy files (**bbpress_2.rb + base.rb**) from my desktop (host) to container :\nsee stackoverflow :[copying-files-from-host-to-docker-container](http://stackoverflow.com/questions/22907231/copying-files-from-host-to-docker-container)\n\n```\n# desktop as root\ndocker ps # to find the \"CONTAINER ID\"\ncp *.rb /var/lib/docker/devicemapper/mnt/_CONTAINER_ID_follow_by_some_ hex/rootfs/var/www/discourse/script/import_scripts\n```\nAnd I use up arrow (last command shell history) to update file.\n\n```\n# Container\nsu discourse\ncd /var/www/discourse\n\n#RAILS_ENV=production bundle exec ruby script/import_scripts/bbpress_2.rb\nRAILS_ENV=production ruby script/import_scripts/bbpress_2.rb bbcode-to-md\n```\n\nIf you change the method redirect(oldpath, post_id) to collect rules instead of making redirection, then you have to import redirection via script (generated by bbpress_2.rb or other):\n```\n# Container\nsu discourse\ncd /var/www/discourse\n\nRAILS_ENV=production rails c \u003c/tmp/bbpress_redirection.rb\n```\n\n## Post import\n\nWhen happy with the import's result:\n\n1. backup in discourse\n2. check that backup is not in the container, but on the host hdd (available when container is stopped)\n3. restore the backup into your target (production) discourse instance OR into the rebuilded discourse container to have a clean install.\n4. personnalize, customize settings\n  * visual: colors, logo, css, ...\n  * badges, categories, groups, ...\n  * ...\n5. backup again to keep customization\n\n## Commands\n\n*  to reset your database\n  Grant superuser rold to discourse account in postgres to avoid issues like :\n\n  * Peer authentication failed for user \"discourse\"\n  * permission denied to create extension hstore (or pg_trgm )\n\n  ```\n  # Container as root\n  su postgres\n  psql -c 'ALTER ROLE discourse SUPERUSER'\n  exit\n  ```\n\n  Reset the db with service stopped\n  ```\n  # Container as root\n  sv stop unicorn  # stop discourse service inside the container\n  su discourse\n  cd /var/www/discourse\n  RAILS_ENV=production bundle exec rake db:drop db:create db:migrate\n  exit\n  sv start unicorn\n  ```\n\n* to activate an account without \"sending activation email\" (or using smtp server)\n  ```\n  # Container as root\n  cd /var/www/discourse\n  su discourse\n  RAILS_ENV=production bundle exec rails c\n\n  irb\u003e u = User.find_by_username_or_email('myemailaddress@me.com')\n  irb\u003e u.password = \"azertyuiop\"\n  irb\u003e u.activate\n  irb\u003e u.admin = true\n  irb\u003e u.save\n  ```\n\n  Wait few minutes, try to log-in on web.\n  If discourse say \"blabla activation email\", wait, reload page, retry (you can do  som ` u.activate`)\n\n* Rewrite rules for nginx, to redirect your old/previous url\n  In the container, try rules by editing `/etc/nginx/conf.d/discourse.conf` and restart nginx via 'sv restart nginx'.\n  But to keep rules over a container rebuild, you should create a template in `discourse_docker/templates/web.rewriteurl.template.yml` (like the file in this project)\n  ```\n  params:\n\n  run:\n    - replace:\n       filename: \"/etc/nginx/conf.d/discourse.conf\"\n       from: /server.+{/\n       to: |\n         server {\n           location /forum/ {\n             rewrite ^/forum/topic/([^/]*)/.*$ /t/$1/ permanent;\n             rewrite ^/forum/.*$ / permanent;\n             return 403;\n           }\n           rewrite ^/wiki/(.*)$ http://wiki.jmonkeyengine.org/$1 permanent;\n  ```\n  Then add this file in the list of template in `discourse_docker/containers/app.yml`\n  ```\n  templates:\n    - \"templates/postgres.template.yml\"\n    - \"templates/redis.template.yml\"\n    - \"templates/web.template.yml\"\n    - \"templates/sshd.template.yml\"\n    - \"templates/web.ratelimited.template.yml\"\n    - \"templates/web.rewriteurl.template.yml\"\n  ```\n  Now you can rebuild\n  ```\n  cd discourse_docker\n  ./launcher rebuild app\n  ```\n\n# Links\n\n* https://meta.discourse.org/t/how-to-run-an-import-script-in-docker/21599/8\n* https://meta.discourse.org/t/paid-need-a-vanilla-2-import-tool/14852/23\n* https://meta.discourse.org/t/re-importing-data-after-migration/22058/2\n* https://meta.discourse.org/t/advanced-troubleshooting-with-docker/15927\n* https://meta.discourse.org/t/redirecting-old-forum-urls-to-new-discourse-urls/20930\n* https://meta.discourse.org/t/advanced-troubleshooting-with-docker/15927\n* https://meta.discourse.org/t/activate-user-on-a-non-configurated-email-server/9494\n* https://meta.discourse.org/t/programmatically-adjusting-color-variables-with-sass/18332 to choose your colors\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdavidb%2Fforum-migration","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdavidb%2Fforum-migration","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdavidb%2Fforum-migration/lists"}