{"id":15719569,"url":"https://github.com/patrickkidd/ci_vectors","last_synced_at":"2025-06-24T18:39:08.203Z","repository":{"id":181294751,"uuid":"666540530","full_name":"patrickkidd/ci_vectors","owner":"patrickkidd","description":"Experiments in calculating emotion as vectors for collective intelligence","archived":false,"fork":false,"pushed_at":"2023-07-15T17:10:37.000Z","size":474,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-30T22:44:25.715Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-2-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/patrickkidd.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-07-14T19:37:46.000Z","updated_at":"2023-07-14T19:49:47.000Z","dependencies_parsed_at":"2024-10-24T15:29:09.854Z","dependency_job_id":"1f005ac2-4039-4c01-95c5-529aa0b9dc63","html_url":"https://github.com/patrickkidd/ci_vectors","commit_stats":null,"previous_names":["patrickkidd/ci_vectors"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/patrickkidd/ci_vectors","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/patrickkidd%2Fci_vectors","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/patrickkidd%2Fci_vectors/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/patrickkidd%2Fci_vectors/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/patrickkidd%2Fci_vectors/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/patrickkidd","download_url":"https://codeload.github.com/patrickkidd/ci_vectors/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/patrickkidd%2Fci_vectors/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":261735152,"owners_count":23201966,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-03T21:56:09.598Z","updated_at":"2025-06-24T18:39:08.182Z","avatar_url":"https://github.com/patrickkidd.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# ci_vectors :: Collective Intelligence Vectors\n\nThis is an experiment in deriving collective opinions from a list of diverse\nopinions on a topic. The motivation is a key theoretical principle in the\nquantitative field of Collective Intelligence, that the average of a set of\ndiverse viewpoints on a single topic is often more accurate to reality than a\nsingle expert. A goal here is to take a list of english-language opinions, in\nthis case from a single Twitter thread, and produce some aggregated summary of\nthe opinions from it.\n\nSee \"Theoretical Background\" for more.\n\nI don't have any experience in Language Models or machine learning, and am not\nthat great with data science in general. I do have a doctorate in Psychology and\nadvanced training in Bowen Theory, which guides some of the novel propositions\nto augment findings from the field of Collective Behavior.\n\n## Experiment 1: Twitter thread replies\n\nA script was written to scrape twitter thread replies from a safari dev tools\nnetwork tab HAR (JSON) export of all requests. Reply text are written to a json\nfile. This is the raw inputs for this project. A little over 200 replies can be\nproduced after manually scrolling to the bottom of a twitter thread to load them\nall asynchronously. Twitter appears to cap the results provided in the web\ninterface.\n\nThe idea is that the thread's original tweet is the \"topic.\" All replies in that\nthread are assumed to be \"votes\" on that topic. A goal is to start with tweets\nfrom public figures that generate many thousands of polarized replies for every\ntweet, like @jordanbpeterson or @elonmusk.\n\n_Opinion Summary_\n\n`test_summary.py` uses the ChatGPT API to produce an\nenglish-language summary of all of the opinions. This requires the number of\nopinions to be capped at about 175.\n\nChatGPT thread: https://chat.openai.com/share/78b0cde6-6374-4ef3-bb52-20f860ee8e0f\n\n_Factor Analysis_\n\n`test_factors.py` uses `sklearn` and `nltk` to generate a\nfactor analysis of terms. This script has not been tested yet. The idea here is\nthat instead of a single \"average\" of the opinons, that opions might cluster\ntogether with different weights. That information might be useful for a person\nto aggregate the factors subjectively in their head to draw a single conclusion.\n\nChatGPT Thread: https://chat.openai.com/share/7e9ba27d-6138-4d11-a6ee-100e2c10d3a3\n\n_Other Ideas_: \n\nHere are some other attempts at getting starting points from ChatGPT:\n\n- Emotion Vectors 1: https://chat.openai.com/share/99691cdb-9d83-4a8b-b2d5-7910a6d70f42\n- Emotion Vectors 2: https://chat.openai.com/share/009f720d-2da4-4fe2-90fc-cea21cb9986d\n- Straight CI: https://chat.openai.com/share/f7564a4f-e287-4518-80e2-372bd0910fdc\n\n\n## Theoretical Background\n\nA current goal of Twitter is to become the \"least untrue\" source of information\navailable. The key assumption is that if individuals are allowed to express\ntheir opinions as freely as possible on a given topic, then it will be the best\nfor society's ability to make sense of that topic. It turns out that there is a\ntechnical argument for this assumption in Collective Intelligence, which\nsuggests that biological systems have evolved instinctual mechanisms for making\ngroup decisions that are far superior to a decision that any one individual\ncould make. These intelligience mechanisms work through local interactions\nbetween relatively uninformed individuals that aggregate into a single and far\nmore sophisticated decision making system.\n\nThis principle applies to baboons making pathfinding decisions, bees making\ndecisions on where to move a nest, schools of fish avoiding a predator in a\nmanner that minimizes individual losses, and others. In the case of humans, this\npertains to the automatic verbal and non-verbal cues that occur when multiple\nstakeholders engage with each other on problem. This instinctual dialog, or\n*dialogos* in Latin, can hypothetically occur in debates, arguments, even wars.\n\nA new hypothetical idea is that automatic emotional responses in humans during\ndialog represent \"votes\" on the problem. The votes are automatically tallied\nthrough the actual behavioral outcome of the dialog. All of this is so automatic\nthat we are not used to observing it while it happens.\n\nFor example, consider that persons A, B, and C are trying to figure out where\nhold an office Christmas party. The dialog might look like this:\n\n- Person A: \"Let's have it at the roller rink!\"\n- Person B: \"That place is too small.\"\n- Person A: \"But they have the best activity that everyone can do.\"\n- Person C: \"But their food is terrible.\"\n- Person B: \"How about just having it at the office?\"\n- Person C: \"I guess that could work!\" (with great positivity)\n- Person A: \"Hmmm, Let me think about it.\"\n- Person B: \"It would be free, and everyone knows how to get there.\"\n- Person C: \"Yeah, com on Person A, let's do it there.\"\n- Person A: \"Well, I'm not totally on board but it's two against one.\"\n- Person B: \"OK, sorry but I think it makes the most sense.\"\n- The Christmas Party ended up at the office.\n\nIn this sequence each interaction is a vote that biases the final decision in a\nparticular direction. For example, \"Let's have it at the roller rink!\" is a\nclear vote for the roller rink. \"That place is too small\" is clear vote against\nthe first vote. There are more votes that push the group's bias toward or away\nthe roller rink. Then a new vote for the office comes, a few more votes for and\nagainst the office, and one with great positivity that carries a bit more weight\nthan the others.\n\nFinally, the actual behavioral outcome is to have the party at the office. The\nactual behavioral outcome is important because it may not always be the outcome\nthat was explicitly agreed upon. The main idea of CI is that the mechanism for\ncollective intelligence has *evolved*, which means it is instinctual/automatic,\nwhich means that it may not be the same as what people say they will do. People\ndo not always do what they say they will do and *instinctual* behavior can only\nbe inferred through actual behavior.\n\nSimilarly, the decision was not unanimous. This is important. Agreement is not\nrequired, and it is epected that there will be some votes that are not in line\nwith an explicit group decision or actual behavioral outcome. This is because\nthe CI process is not about *fairness*, it is about getting as close to reality\nas possible. The key CI principle is that each individual provides a vote or\nvotes that merely biases the decions in one direction or another. The \"votes\"\nare then tallied and then the group decision is enforced through actual\nbehavior.\n\nAnecdotal evidence for this process existing at all is that each step in a\ndialog series has an impact on each individual's model of the problem. People\nare heavily influenced by the process of deliberation in a group. Exactly *how*\nthis occurs is unknown and likely very complex.\n\nHowever, evidence *that* it occurs can be obtained in a single experiement. Ask\neach person in an experimental group for their opinion on a topic separately\nprior to being told they will participate in a group deliberation on that topic.\nThen hold the group deliberation, recording each statement a person makes to the\nothers about the topic. The prediction is that at least some of the people will\ndemonstrate that their opinions have changed while the deliberation is occuring,\neither verbally or non-verbally. This only occurs after seeing or hearing the\nresponses from others.\n\nMost people do not really own their opinons, as demonstrated in this kind of\nexperiment, and more formally in this experiment where the evolution of\nindividual opinions through group context was demonstrated:\nhttps://www.youtube.com/watch?v=O_m8IZzl48E\u0026feature=youtu.be. Confederates could\nflip an individual opinion on a topic merely by flipping their own. The\nproposition here is that individual's opinions on a problem exist and operate\nprimarily in conjunction with the group of affiliated stakeholders on that\nproblem, and that the end result is something like an average of the individual\nopinions.\n\nA hypothesis is that this applies even when groups are highly polarized on a\ntopic. In fact, the proposition is that the poles *function* to balance each\nother out in this instinctual CI process. In other words, in an argument the\n\"truth is somehwere in the middle.\" Providing a method for producing an opinion\n\"somewhere in the middle\" from a list of diverse opinions would benefit\ncollective decision making, especially on difficult topics.","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpatrickkidd%2Fci_vectors","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpatrickkidd%2Fci_vectors","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpatrickkidd%2Fci_vectors/lists"}