{"id":16572772,"url":"https://github.com/lemire/programmingishard","last_synced_at":"2026-03-07T12:36:50.963Z","repository":{"id":66054914,"uuid":"110028064","full_name":"lemire/programmingishard","owner":"lemire","description":"Long-term book project","archived":false,"fork":false,"pushed_at":"2021-10-30T17:12:56.000Z","size":41,"stargazers_count":34,"open_issues_count":1,"forks_count":2,"subscribers_count":8,"default_branch":"master","last_synced_at":"2025-03-05T14:29:20.759Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/lemire.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2017-11-08T20:40:22.000Z","updated_at":"2023-05-21T19:39:53.000Z","dependencies_parsed_at":null,"dependency_job_id":"2d800276-2633-4e97-84f6-262502ea393f","html_url":"https://github.com/lemire/programmingishard","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/lemire/programmingishard","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lemire%2Fprogrammingishard","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lemire%2Fprogrammingishard/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lemire%2Fprogrammingishard/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lemire%2Fprogrammingishard/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/lemire","download_url":"https://codeload.github.com/lemire/programmingishard/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lemire%2Fprogrammingishard/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":30213260,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-03-07T12:15:00.571Z","status":"ssl_error","status_checked_at":"2026-03-07T12:15:00.217Z","response_time":53,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-11T21:28:33.225Z","updated_at":"2026-03-07T12:36:50.931Z","avatar_url":"https://github.com/lemire.png","language":null,"funding_links":[],"categories":[],"sub_categories":[],"readme":"# programmingishard\n\nCurrent status: a book project is ongoing but this repository will not be updated in the short term.\n\n## Message to random visitors :\n\nUltimately, I plan for this GitHub repository to turn into a \"book\".\nSomewhat accidentally, I made the repository public, and some people\ntook notice. \n\nI have since decided to leave the repository public.\n\nYet, this project not nearly ready for public consumption. It is a mess.\nIt is naive. It is boring.\n\n\n## Preface\n\nI learned to program when I was barely twelve years old. Like most kids,\nI started out trying to program my own video games. And I largely failed.\nThe projects would start out well, but then I would often encounter surprising\nproblems. \n\nFor example, I recall learning how to draw a strawman, pixel by \npixel (at the time, in the 1980s, we had few pixels). It worked beautifully.\nSo I figured that animating the strawman would be easy. Just draw the strawman,\nclear the screen, and redraw the strawman. On paper, it is a flawless plan.\nIn practice, the result was horrible. It kept my processor fully occupied\nwhile the screen was flashing.\n\nIn a later project, I decided to stick with moving a \"tank\" made of only\na couple of pixels. This worked much better. One of the pixel would flash\nannoyingly, but I decided it was a feature. Then I decided that my tank should\nfire bullets. After a time, I managed to have the \"tank\" fire a single\nbullet (made of a single pixel). It worked beautifully... with the bullet\nflying through the screen. At first, the bullet would mave at the same speed\nas the tank, so one pixel at a time. But that was a problem because the tank\ncould litterally keep up with its bullet. So I doubled the speed of the bullet,\nso it could move two pixels with each time increment. This was better, but not\nfast enough. I think I ended up making the bullet move four pixels at a time.\nMuch better. Then I added walls to the environment. My walls were one pixel \nthick, naturally. I did not have to redraw them when the tank moved, but I wanted\nboth the tank and the bullets to be stopped by the walls.  Because the walls\nhad a different color, I found it easy to prevent the tank from moving into a wall:\njust check whether the color pixel. With the bullets, it was something else:\nthey moved four pixels at a time... so they could materialize beyond a wall,\nand checking the color of one pixel was no longer sufficient. I kept going and\nfixed many problems. When a friend of mine came over, I invited him to play.\nHe quickly found many vexing problems. For example, by moving in diagonal, it\nwas somehow possible to move the tank through a wall. My friend also noticed\nthat my tank could only fire one bullet at a time. At that point in time,\nmy code was getting a bit large. I could not identify the bugs. I tried\nchanging the code in various ways, but nothing did it. Furthermore, I had\nno idea how allow the tank to shot an unlimited number of bullets, though I did\nfind a way to fire up to two bullets.\n\nOver time, I got a lot better. As a seniors in high school, we had a \ncomputer science class and I could run circles around most (but not all)\nof my peers. I wrote an adventure game with minimalist natural language\nprocessing. I wrote an Othello board game.\n\nI then took my first serious programming class, at a local community \ncollege. I learned to program in Pascal. I discovered IDE (integrated\ndevelopment environments), debugers, structured programming and so forth.\nI believe that the class I took was commonly taken by students who were\npursuing technical studies. Though it was an introductory course,\nI was impressed by how little I knew starting in the course.\n\nI never took another programming course. In fact, I never took any \ncomputer science course. While in college, I audited an introductory \ncourse, but it was taught by  lowly graduate students with minimal \nsupervision from a bored professor.\n\nSo, as a programmer, I am largely self-taught. But I expect that it is\nquite common. In the best of cases, college programs have a small handful\nof programming classes. And they are most often in an introductory setting,\na stepping stone to get students to move on to the more important things like \nartificial intelligence, software engineering or theoretical computer science.\n\nIt is maybe for the best. I don't think we learn to program by watching\nsomeone talk in front of a classroom. I think it is a bit ridiculous to\nexpect students to \"program\" with pencil and paper, as if they were Ada\nLovelace. It is also hard to teach what you do not know, or do not love,\nand, evidently, to many computer scientists, programming is a lowly task.\n\nStill, people learn to program. And, collectively, we get better at it. The\n\"trade\" of programming has become much more sophisticated since the days I \nfirst wrote my naive tank game.\n\nAs for myself, I learned a lot from both my mistakes and my successes. Here\nare a few of my early experiences:\n\n- I was tasked by a Physics professor to program a \"galaxy\" simulator in Pascal.\n  One simply had to spray a few stars, give them a nudge, and see how gravity \n  made them move. My simulator was initially quite buggy, but it eventually \n  did display the stars. At each time increment, I had to account for each star\n  \"pulling\" on each other star. Alas I quickly realized that my program did not scale.\n  I could simulate two or three stars, but no matter what I did, I could not make\n  hundreds of stars flying over the screen. Quite simply, as the number of stars\n  increase, the number of interactions between stars increase much faster (quadratically).\n  Thus I learned a valuable lesson: though my computer is fast, it cannot solve\n  all problems quickly.\n- In college, my friends like to play a \"solitaire\" game where one starts with a board\n  where all cells but one is occupied. You can move the pieces like in a Checker game,\n  jumping over another piece. The piece that gets jumped over is discarded. The goal\n  is to empty the board so that a single piece is left. My friends learned all sorts\n  of clever ways to solve this game, and they challenged me. I was annoyed and I did\n  not want to bother learning to play well. So I went to my room, and within an hour\n  or so, I wrote a program that could generate as many solutions as one wanted. My \n  algorithm was super simple: it simply started each new game with a random allowable\n  move. It would keep playing randomly. When no more move was possible, it would check\n  the number of pieces left. If a single piece was left, it would print out the result.\n  This simple randomized algorithm turned out to be very fast. My program was very simple.\n  My friends quickly stopped playing afterward.\n- When Microsoft released Windows 3.1, I was in love. My PC had this beautiful interface.\n  What could I do with it? I decided that I would write a talking clock. It was easy\n  enough. Since we did not have voice synthesizers, I simply recorded my own voice.\n  My talking clock was beautiful! The only problem was that it was large and used almost\n  all the memory in my PC!\n\nOne experience after another, I learned to program. It was difficult.\n\nUsing as credentials my Ph.D. in Mathematics, I became a professor in Mathematics,\nthen a government researcher in \"information technology\", and later a professor\nin computer science. I became a professor somewhat by accident. It is likely\nthat had there been other paths open to me, I might have become a programmer.\n\nNevertheless, I kept being surprised at how little the people I met knew about\nprogramming. You'd think that computer scientists would show some interest\nin the art of programming, but, as a matter of pride, computer scientists would\ninsist that all the programming in their research projects was done by students.\n\nThe implication is clear: programming a low-status activity.\nIt is not uncommon for programmers to work for non-programmers \nwho assume that programming is easy. Many sentences\nstart with \"you just have to...\" and finish with a task that is far\nbeyond the abilities of most people. \"No, you cannot just convert these\nscanned pages into your favorite word processor format without introducing\nerrors.\"\n\nEducators have tried for years to make programming sound easy. You\nhave great projects like Scratch. Many business leaders promote software\nprogramming as a career.\n\nYou would think that I would be pleased at such development, and in some\nway I am. \n\nI used to think that important programming projects\nrequired large teams with much funding, or few exceptional people.\nI no longer think so. Advanced programming skills can be acquired,\nthey can even be acquired quickly. That's because programmers are\nan ingenious bunch. More than maybe any other professional community,\nthey have built tools and made available their know-how, all of it\npackaged in a particularly accessible way, if only you know where to look.\n\nBut I fear that part of these initiatives are contributing to\nthe myth that programming is a shallow skill. I don't think it is true. \nProgramming is hard in the same way that writing, machining, dansing,\nor baseball is hard.\n\nWe used to think that most people could never learn to read and write.\nWe now know that most people can learn to read and write. However, few\npeople can write for the New York Times. Almost nobody can become a famous\nnovelist. Writing well is hard.\n\n\n\n\n\n\n## random plan\n\n\nbeyond naive theory:\n  - models lie: constant-time are not constant time, focus on the data size you have\n  - hash tables can fail you\n  - maps and sets : hash tables are not O(1), often not even close\n\nmemory is hard \n  - throughput and latency\n - locality matters \n  - objects consume memory: fewer objects, less memory\n\nmemory is parallel, so even harder\n - little’s law\n  - memory access is parallel, and this is limited by instruction windows, batching is good\n  - extra core/parallel code can be necessary to achieve best throughput\n\n\nYour processor can do many things *per* cycle\n  - Your processor can see in the future\n  - Unpredictable branches are bad\n  - this interacts either favorably or unfavorably with memory: linked list is bad  \n\nnumbers are hard\n  - most calculators can't do 10000000000000000 + 1 = 10000000000000001\n  - floating point are difficult a*b  - a * b is not always zero, often not even close\n  - subnormal \n  - NaN\n  - associativity\n  - two's complement: how are negative values represented?\n\nstrings are hard\n  - Unicode, UTF-8, UTF-16, what is a \"character\", how many \"characters\" in your string? How do  you represent an emoticon?\n  - Sorting, hashing\n\nrandomness is hard\n- generating good random numbers can be hard\n- useful algorithms: Bloom filters, reservoir sampling, Knuth's shuffle.\n\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flemire%2Fprogrammingishard","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flemire%2Fprogrammingishard","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flemire%2Fprogrammingishard/lists"}