Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/neelsoumya/special_topics_unconventional_ai

Special topics class on unconventional AI
https://github.com/neelsoumya/special_topics_unconventional_ai

artificial-intelligence artificial-intelligence-algorithms computer-science deep-learning machine-learning teaching-materials unconventional-ai

Last synced: 4 days ago
JSON representation

Special topics class on unconventional AI

Awesome Lists containing this project

README 

        
# special_topics_unconventional_AI



## Introduction



A reading list for a special topics class (R255) at the Computer Science and Technology Department at the University of Cambridge. This is part of Advanced Topics in Machine Learning.



The title of the course is: 



`Unconventional approaches in AI: complex systems perspectives, cognitive psychology, social sciences, computational models of creativity, explainable AI inspired by other disciplines and other unconventional models`



This is AI or classical AI before big data. The time is now ripe to revisit these wonderful ideas and think about how to incorporate them in modern AI/deep learning. Insights from the past can inform future approaches to AI, especially in the age of big data.



Looking at the heritage of computing and its interdisciplinary past can inspire new approaches for the future. We need to learn lessons from the history of AI, what approaches worked and did not work in the past and how AI went through multiple `winters`.

	

These approaches can be used to develop techniques that can inspire explainable AI.



### Talk describing this topic



* Short introductory talk



https://www.youtube.com/watch?v=o7EXf265sTU



* Full talk



https://youtu.be/8s4vVPTGVfw



* Slides



https://github.com/neelsoumya/special_topics_unconventional_AI/blob/main/intro.pdf



https://github.com/neelsoumya/special_topics_unconventional_AI/blob/main/wrapup.pdf



### Cognitive psychology, computational models of creativity and other unconventional models in AI



* Computational models of creativity and scientific insight (Pat Langley)



https://escholarship.org/content/qt54x8v354/qt54x8v354.pdf



* BACON.5: the discovery of conservation laws (Pat Langley)



https://dl.acm.org/doi/abs/10.5555/1623156.1623181



* A Computational Inflection for Scientific Discovery



https://cacm.acm.org/magazines/2023/8/274938-a-computational-inflection-for-scientific-discovery/fulltext



* Copycat (a computational model of analogy making)



https://dspace.mit.edu/handle/1721.1/5648



* The emergence of understanding in a computer model of concepts and analogy-making



https://www.sciencedirect.com/science/article/abs/pii/0167278990900865?via%3Dihub



* Structure mapping engine: Algorithms and examples



https://doi.org/10.1016/0004-3702(89)90077-5



* Learning new principles from precedents and exercises (Winston)



https://doi.org/10.1016/0004-3702(82)90004-2



* The overfitted brain: Dreams evolved to assist generalization



https://doi.org/10.1016/j.patter.2021.100244



* DreamCoder: Growing generalizable, interpretable knowledge with wake-sleep Bayesian program learning



https://arxiv.org/abs/2006.08381



* On the measure of intelligence (Abstraction and Reasoning Corpus)



https://arxiv.org/abs/1911.01547



* Commonsense reasoning



https://dl.acm.org/doi/10.1145/2701413



Commonsense reasoning, Cyc and large language models



https://arxiv.org/pdf/2308.04445.pdf



Cyc database of commonsense reasoning (Doug Lenat and Gary Marcus)



http://web.archive.org/web/20230902080842/https://garymarcus.substack.com/p/doug-lenat-1950-2023



* Stories and narratives (Patrick Winston)



https://nautil.us/the-storytelling-computer-8380/



https://dspace.mit.edu/handle/1721.1/67693



* Argument technology for debating with humans



IBM Project Debater



https://www.nature.com/articles/d41586-021-00539-5



* Case based reasoning (Janet Kolodner)



http://alumni.media.mit.edu/~jorkin/generals/papers/Kolodner_case_based_reasoning.pdf



### Papers from philosophy and consciousness studies



Can we (and should we) have consciousness in machines?



* Consciousness in Artificial Intelligence: Insights from the Science of Consciousness



https://arxiv.org/pdf/2308.08708.pdf



* Levels of AGI: Operationalizing Progress on the Path to AGI



https://arxiv.org/pdf/2311.02462.pdf



## Papers related to cognitive architectures



* 40 years of cognitive architectures: core cognitive abilities and practical applications



https://link.springer.com/article/10.1007/s10462-018-9646-y



### Papers related to understanding and reasoning in large language models (LLMs): The philosophy and psychology of large language models



* Can large language models reason or understand?



https://arxiv.org/pdf/2210.13966.pdf



* Are they capable of consciousness?



https://www.economist.com/by-invitation/2022/09/02/artificial-neural-networks-are-making-strides-towards-consciousness-according-to-blaise-aguera-y-arcas



* World models in large language models



https://arxiv.org/pdf/2310.02207.pdf



* Nature review



https://www.nature.com/articles/d41586-023-02361-7



* Sparks of Artificial General Intelligence: Early experiments with GPT-4



https://arxiv.org/abs/2303.12712



* A Theory for Emergence of Complex Skills in Language Models



https://arxiv.org/pdf/2307.15936.pdf



* LLMs and the Abstraction and Reasoning Corpus: Successes, Failures, and the Importance of Object-based Representations



https://arxiv.org/pdf/2305.18354.pdf



* Large Language Models as General Pattern Machines



https://arxiv.org/abs/2307.04721



* Hubert Dreyfus's crtique of Winograd's block worlds program



https://archive.org/details/whatcomputerscan017504mbp/page/n39/mode/2up



* Can LLMs understand? (Far from being “stochastic parrots,” the biggest large language models seem to learn enough skills to understand the words they’re processing.)



https://www.quantamagazine.org/new-theory-suggests-chatbots-can-understand-text-20240122/



* Large Linguistic Models: Analyzing theoretical linguistic abilities of LLMs



 https://arxiv.org/abs/2305.00948



* Theory of Mind benchmark for large language models



  https://arxiv.org/abs/2402.06044



  https://github.com/seacowx/OpenToM



* Are Emergent Abilities of Large Language Models a Mirage?



  https://arxiv.org/abs/2304.15004



* A Philosophical Introduction to Language Models



  https://arxiv.org/abs/2401.03910



* Machine Psychology



  https://arxiv.org/abs/2303.13988  



## Artificial Stupidity



* Lessons for artificial intelligence from the study of natural stupidity



https://www.nature.com/articles/s42256-019-0038-z



### Papers related to explainable AI (xAI) inspired by other disciplines



* Explanation in artificial intelligence: Insights from the social sciences (Miller)



https://arxiv.org/abs/1706.07269



### Collective intelligence



* Collective intelligence for deep learning: A survey of recent developments



https://journals.sagepub.com/doi/full/10.1177/26339137221114874



*  Eric W. Bonabeau. Control mechanisms for distributed autonomous systems: insights from social insects. 2001



In Design Principles for the Immune System and Other Distributed Autonomous Systems.



https://www-users.cs.york.ac.uk/susan/books/pages/s/LeeASegel.htm#9582



(login with your RAVEN ID and search the university library webpage)



https://idiscover.lib.cam.ac.uk/



* Neural cellular automata



https://distill.pub/2020/growing-ca/



### Reading science fiction and understanding the philosophy and ethics of AI



Some papers and readings on using science fiction to understand the philosophy and ethics of AI.



1. The Mind's I: Fantasies and reflections on self and soul. By Douglas Hoffstadter and Daniel Dennett



2. https://cacm.acm.org/research/how-to-teach-computer-ethics-through-science-fiction/



### Miscellaneous Resources



* Analogy in AI



https://melaniemitchell.me/PostdocProjectDescription.pdf



* Abstraction and Reasoning Corpus



https://github.com/fchollet/ARC



* Abstraction and Reasoning Corpus Challenge



https://blog.jovian.ai/finishing-2nd-in-kaggles-abstraction-and-reasoning-challenge-24e59c07b50a



https://github.com/alejandrodemiquel/ARC_Kaggle



> Domain specific languages may be required (as suggested by Chollet) like genetic algorithms and cellular automata



* Other paths to intelligence elsewhere in the animal kingdom



https://nautil.us/another-path-to-intelligence-23113/



* The storytelling computer



https://nautil.us/the-storytelling-computer-237502/



http://web.archive.org/web/20221102094120/https://nautil.us/the-storytelling-computer-237502/



* The Psychology of Invention in the Mathematical Field (Jacques Hadamard)



https://archive.org/details/eassayonthepsych006281mbp/page/n35/mode/2up



* The Theory of Mind



https://en.wikipedia.org/wiki/Theory_of_mind



* The structure of scientific revolutions, Kuhn



### Code repositories and implementations



https://github.com/Tijl/ANASIME



https://github.com/crazydonkey200/SMEPy



https://github.com/fargonauts/copycat



### Presentations on selected papers



Present and lead a discussion on one of these papers (or any other related paper: come speak with me).

The idea is that you raise some interesting questions. This course is meant to teach you research skills (like thinking critically about a paper and literature review skills).



### Writeup



In this course, each student would chose one paper. They would then do a presentation on it. 



Towards the end of the term they would do a writeup/short report: 



* on this paper, and 



* the topic in general (unconventional AI). They would do a literature review of other papers in the field. 



* They will then reflect/write on how these techniques can be incorporated in modern AI/deep learning. 



The intention is for students to learn how to read papers, and compare and contrast them to other papers and then evaluate what this means for AI/deep learning.



Some writing prompts for the writeup are here:



* Short report (less than 4000 words). The idea is write a coherent narrative. 



* Suggest how these ideas can be incorporated in modern AI/deep learning systems



* Why do you think these ideas were not successful in the 1950s/1960s?



* What kind of data would we need to ensure these techniques would work today?



* What lessons can we learn from the history of AI, what approaches worked and did not work in the past?



* What could be the disadvantages of these approaches?



* Rational reconstruction (analytical literature review/survey) of a research area 



Other thoughts on the writeup:



* A detailed research proposal with some ground work already accomplished



* A hybrid of all of the above



Thoughts on a project:



* If you feel ambitious, you can do a coding project based on open source data (talk to me if you would like to do this).



### Administrivia



https://www.cl.cam.ac.uk/teaching/2122/R255/



https://github.com/neelsoumya/special_topics_unconventional_AI/blob/main/admin_notes.md



### Miscellaneous



* How to read papers



https://www.cs197.seas.harvard.edu/



https://docs.google.com/document/d/1bPhwNdCCKkm1_adD0rx1YV6r2JG98qYmTxutT5gdAdQ/edit#heading=h.yxlvj6bo3y2



* How to write



Write regularly



Keep a schedule



https://sites.google.com/site/neelsoumya/research-resources/scientific-writing



Video on writing



https://www.youtube.com/watch?v=DeVjXINr5Wk



Book on writing (please contact me to borrow a copy; also available from the library digitally)



`How to Write a Lot: A Practical Guide to Productive Academic Writing` by Paul J Silvia



### Contact



You can also pick other papers that are broadly in this area/topic and that excite you. Please contact me to discuss further.



Soumya Banerjee



[email protected]



[email protected]



Office: FC01 (Computer Science and Technology Department)



https://sites.google.com/site/neelsoumya/Home



https://github.com/complexsystemslab/project_ideas/blob/main/project_ideas.md