{"id":20904964,"url":"https://github.com/zukarusan/jchoreco","last_synced_at":"2025-05-13T05:30:56.878Z","repository":{"id":113606598,"uuid":"413357317","full_name":"zukarusan/JChoreco","owner":"zukarusan","description":"Java automatic chord recognition with neural network and digital signal processing.","archived":false,"fork":false,"pushed_at":"2022-08-28T03:02:22.000Z","size":1817,"stargazers_count":11,"open_issues_count":7,"forks_count":2,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-04-01T18:16:31.201Z","etag":null,"topics":["audio-processing","chord-recognition","chromagram","deep-learning","java","java-api-tensorflow","keras-tensorflow","machine-learning","music","neural-network","signal-processing","stft","tarsos","tarsosdsp","tensorflow"],"latest_commit_sha":null,"homepage":"","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/zukarusan.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-10-04T09:31:41.000Z","updated_at":"2024-05-27T20:26:01.000Z","dependencies_parsed_at":null,"dependency_job_id":"4735495d-f4df-4a9d-8e02-3be054907756","html_url":"https://github.com/zukarusan/JChoreco","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zukarusan%2FJChoreco","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zukarusan%2FJChoreco/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zukarusan%2FJChoreco/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zukarusan%2FJChoreco/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/zukarusan","download_url":"https://codeload.github.com/zukarusan/JChoreco/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":253882795,"owners_count":21978548,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["audio-processing","chord-recognition","chromagram","deep-learning","java","java-api-tensorflow","keras-tensorflow","machine-learning","music","neural-network","signal-processing","stft","tarsos","tarsosdsp","tensorflow"],"created_at":"2024-11-18T13:20:01.428Z","updated_at":"2025-05-13T05:30:56.872Z","avatar_url":"https://github.com/zukarusan.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Java Chord Recognition\n\n---\n\nA java musical chord recognition utility using pre-trained neural network Tensorflow model and DSP tools extended from [TarsosDSP][1]. \n\n\u003e ***WARNING***: This library is still in development that has unorganized source code and some features are not implemented on top of another\n\nThe **main processing tools** in package dir you might want to use are:\n\n- `Chord Processor`: A sub-*TarsosDSP* class extended from [`AudioProcessor`][3] that wraps Tensorflow model and\n  other main processing components\n- `Chord Predictor`: A singleton class that predict the chroma given in the params.\n- `CRP Vector Factory`: A factory for creating chroma object, specifically CRP chroma, from samples given.\n\nThis toolkit has currently the following main component classes:\n\n- `Sound File` : A naive Interface for retrieving full samples from WAV or MP3 file.\n- `Signal`: A naive float array of raw sample retrieved from Sound File or from other src\n- `Log Frequency Vector`: A full descriptive class that has pitch information from a sample\n- `Chroma Vector`: Interface class that has currently 3 different types of chroma classes.\n  For the details of these different chroma, please refer to the *Method and Approach* section\n- `Spectrum`: A full description abstract class that has spectral information with the given sub-classed type\n- `Tensorflow Model`: A specific tensorflow model loader and runner with its [Java API][2]. The loaded model is\n  pretrained and stored in main resources package\n  \n## Chord sets and model\n\n---\n\nAvalaible chord sets that can be predicted by the pre-trained neural model is located in the Chord.java class. They are in all keys with only major and minor scale. The chords list given the model output are respectively:\n```java\n{\n    \"A#maj\", \"A#min\", \"Amaj\", \"Amin\", \"Bmaj\", \"Bmin\", \"C#maj\", \"C#min\",\n    \"Cmaj\", \"Cmin\", \"D#maj\", \"D#min\", \"Dmaj\", \"Dmin\", \"Emaj\", \"Emin\",\n    \"F#maj\", \"F#min\", \"Fmaj\", \"Fmin\", \"G#maj\", \"G#min\", \"Gmaj\", \"Gmin\"\n}\n```  \nThe model is set to have ~0.7 testing accuracy with the testing subset consisting of ~400 samples of chroma (specifically CRP) vectors\n\n## Install and Dependencies\n\n---\n### Require min: Java 11\n\n**For easy quick use**:\u003cbr\u003eImport locally the full-bundled fat jar library, [`jchoreco-x.x.x-all.jar`][6], without implementing any dependencies in your code\n(But this might cause big-sized package in your built app)\n\n***Otherwise, for lightweight use***:\u003cbr\u003e\nUse locally the smaller-packaged, [`jchoreco-x.x.x.jar`][7], and the following dependencies must be manually included in your project. \n\n```java\n// gradle example (kotlin)\nimplementation(\"com.github.axet:TarsosDSP:2.4\") // TarsosDSP libraries\nimplementation(\"com.github.wendykierp:JTransforms:3.1\") // Pure java FFT libraries\nimplementation(\"org.tensorflow:tensorflow-core-platform:0.4.0\") // Tensorflow API\nimplementation(\"javazoom:jlayer:1.0.1\") // MP3 Decoder\n```\n\n### Optional dependencies\n```kotlin\nimplementation(\"org.apache.commons:commons-math3:3.6.1\") // For CommonProcessor tools\nimplementation(\"com.github.yannrichet:JMathPlot:1.0.1\") // For plotting using swing ui\n```\n\n## Quick wrap example\n\n---\n\n*For all examples, it is assumed the chord model is loaded in the package.\nRefer to Usage Example for details.*\n\u003cbr\u003e\u003cbr\u003eFrom file, predict chord **cautiously** of *all samples in the file* with the following code\n```java\nString pathFile = \"/path/to/file.wav\";\nSignal signal = new WAVFile(new File(pathFile)).getSamples(0);\n\ntry (ChordPredictor predictor = ChordPredictor.getInstance()) {\n    String chord = predictor.predict(CRPVectorFactory.from_signal(signal));\n    System.out.println(\"Predicted chord: \"+chord);\n}\n```\n## Usage example\n\n---\n\n### Extracting samples\nUse either `SoundFile` or *TarsosDSP*'s [`AudioDispatcher`][4] to retrieve, respectively, full or buffer \nsamples from an audio. For `SoundFile`:\n```java\nFile file = new File(pathString); // java.io.File\n\n// Caution: This retrieves all sample from the file\n\nSoundFile sound = new MP3File(file);\n// or\nSoundFile sound = new WAVFile(file);\n\nSignal signal = sound.getSamples(0); // get signal from channel 1\n// or\nfloat[] samples = signal.getData(); // if you need float samples, retrieve its data of the audio\nfloat sampleRate = signal.getSampleRate();\n```\nFor *TarsosDSP*'s` approach, refer to its [example applications][5]\n\n### Collecting  chroma (CRP) vector\n\nSimply use `CRPVectorFactory` to exctract the chroma vector from samples sources\n```java\nCRP chromaCRP = CRPVectorFactory.from_signal(signal);\n// or\nfloat[] chromaCRP = CRPVectorFactory.from_floatSamples(samples, sampleRate);\n```\n\n### Predicting chord from chroma vector\n\n\u003e ***CAUTION*** : Using this library chord predictor/processor is not thread-safe. This is because the current tensorflow\n\u003e model java api (v0.4.0) declare that some of its resources must be manually released by the caller.\n\u003e \n\u003e To avoid memory leak, use pre-cautiously with try-with-resources block or call `close()` method \n\u003e from the predictor/processor class to release the resources.\n\n\u003e **Alert**: Make sure the pre-trained tensorflow model is provided in the main resource package \n\n#### Using singleton `ChordPredictor`\n\nSurround with try-with-resources block:\n\n```java\n// use try-with-resources block\ntry (ChordPredictor predictor = ChordPredictor.getInstance()) {\n      String chord = predictor.predict(chromaCRP); // predict the chord\n      System.out.println(chord); // printing the chord\n      \n      // or predict list of processed crp\n      \n      List\u003cfloat[]\u003e chromaList = ...;\n      List\u003cString\u003e chords = predictor.predict(chromaList);\n}\n```\nor manually call the `close()`:\n\n```java\n// use with manual close()\nChordPredictor predictor = ChordPredictor.getInstance();\nSystem.out.println(predictor.predict(chromaCRP)); \npredictor.close(); // call the close() whenever finished using it\n```\n\n#### Using `ChordProcessor` and `AudioDispatcher` from TarsosDSP\n\n*TarsosDSP*'s `AudioDispatcher` runs on an audio input stream and provide the sample buffer\nevery once a time defined by the constructor. Adding the `ChordProcessor` to the dispatcher\nis an idea to predict chord in buffer.\n\u003e **ALERT**: `ChordProcessor` is always closed after the dispatcher finished.\n\u003e But it is always a best practice to surround it with try-with-resources block or call the close() again.\n \nTo retrieve the chord bytes of lined string ( the processor output a form of`(chord + '\\n')`) ,\ncreate a class of java `OutputStream` first:\n\n```java\nint BUFFER_SIZE = 1024 * 16; // 16384 bytes or more is good for chord audio buffering\nAudioDispatcher dispatcher = AudioDispatcherFactory.fromFile(file, BUFFER_SIZE, BUFFER_SIZE/2);\n\nOutputStream output = System.out; // Set the output stream as the console\n\ntry (ChordProcessor chordProcessor = \n    new ChordProcessor(SAMPLE_RATE, BUFFER_SIZE, output)) {\n    \n    dispatcher.addAudioProcessor(chordProcessor);\n    dispatcher.run();\n}\n```\n\n### Example applications\n\n\nThere are 2 provided examples in the main package:\n\n- **CLIChordRecognizer**: a chord processing from microphone printing to console. *Warning*:\nInterrupt the program by entering chars, not other keyboard interrupt. \n- **SwingChordRecognizer**: a chord processing from microphone with simple swing GUI.\n\n## Method and Approach\n\n---\n\nThe methodology for this project to achieve chord recognition is based on the following works:\n\n1. Abadi, Martín, Ashish Agarwal, Paul Barham, Eugene Brevdo, Zhifeng Chen, Craig Citro, Greg S. Corrado, et al. 2015. “TensorFlow: Large-scale machine learning on heterogeneous systems,” Software available from tensorflow.org. https://www.tensorflow.org/.\n2. Chollet, François. 2015. “keras.” 2.7.0. GitHub. https://github.com/fchollet/keras.\n3. Chordify B.V. n.d. “About Chordify.” Chordify. Accessed 2022. https://chordify.net/pages/about/.\n4. Fujishima, Takuya. 1999. “Realtime Chord Recognition of Musical Sound: a System Using Common Lisp Music.” 1999. https://quod.lib.umich.edu/i/icmc/bbp2372.1999.446/1.\n5. Garg, Vijay K. 2007. “Pulse Code Modulation.” In Wireless Communications \u0026 Networking. N.p.: Elsevier Science.\n6. Harris, Sarah L., and David Harris. 2021. “3.4.2 State Encodings.” In Digital Design and Computer Architecture, RISC-V Edition. N.p.: Elsevier Science.\n7. Jiang, Nanzhu, Peter Grosche, Verena Konz, and Meinard Müller. 2011. “Analyzing Chroma Feature Types for Automated Chord Recognition.” https://www.researchgate.net/publication/228445808_Analyzing_Chroma_Feature_Types_for_Automated_Chord_Recognition/comments.\n8. Jones, Russell. 2007. Understanding Basic Music Theory. Edited by Catherine Schmidt-Jones. http://cnx.org/content/col10363/1.3/.\n9. Mauch, Matthias, and Chris Cannam. n.d. “Chordino and NNLS Chroma.” isophonics. Accessed 2022. http://www.isophonics.net/nnls-chroma.\n10. Mauch, Matthias, and Simon Dixon. 2010. “Approximate Note Transcription for the Improved Identification of Difficult Chords.” Proceedings of the 11th International Society for Music Information Retrieval Conference (ISMIR 2010). https://www.researchgate.net/publication/220723830_Approximate_Note_Transcription_for_the_Improved_Identification_of_Difficult_Chords.\n11. Microsoft. n.d. “FileDialog Class.” Microsoft Documentation. Accessed January, 2022. https://docs.microsoft.com/en-us/dotnet/api/system.windows.forms.filedialog.\n12. Müller, Meinard. 2007. Information Retrieval for Music and Motion. N.p.: Springer.\n13. Müller, Meinard, and Sebastian Ewert. 2010. “Towards Timbre-Invariant Audio Features for Harmony-Based Music.” IEEE Transactions on Audio Speech and Language Processing 18 (3): 649 - 662. 10.1109/TASL.2010.2041394.\n14. O'Donnell, Michael J. 2004. “Digital Sound Modeling lecture notes for Com Sci 295.” Perceptual Foundations of Sound. http://people.cs.uchicago.edu/~odonnell/Scholar/Work_in_progress/Digital_Sound_Modelling/lectnotes/node4.html.\n15. Oracle Corporation. 2021. “JavaFX Documentation Project.” GitHub Pages. https://fxdocs.github.io/docs/html5/.\n16. Osmalskyj, Julien, Jean J. Embrechts, Marc V. Droogenbroeck, and Sébastien Piérard. 2012. “Neural networks for musical chords recognition.” (January). https://www.researchgate.net/publication/252067543_Neural_networks_for_musical_chords_recognition.\n17. Peeters, Geoffroy. 2006. “Chroma-based estimation of musical key from audio-signal analysis.” (October). https://www.researchgate.net/publication/220723813_Chroma-based_estimation_of_musical_key_from_audio-signal_analysis.\n18. “Pulse-Code Modulation Codec-Filters.” 2018. In The Communications Handbook, edited by Jerry D. Gibson. N.p.: CRC Press.\n19. Reenskaug, Trygve. 1979. “Models-Views-Controllers,” A whitepaper report of MVC. https://folk.universitetetioslo.no/trygver/1979/mvc-2/1979-12-MVC.pdf.\n20. Shah, Ayush K., Manasi Kattel, Araju Nepal, and D. Shrestha. 2019. “Chroma Feature Extraction.” (January). https://www.researchgate.net/publication/330796993_Chroma_Feature_Extraction.\n21. Sheh, Alexander, and Daniel P. Ellis. 2003. “Chord Segmentation and Recognition using EM-Trained Hidden Markov Models.” https://www.researchgate.net/publication/36709060_Chord_Segmentation_and_Recognition_using_EM-Trained_Hidden_Markov_Models.\n22. Six, Joren, Olmo Cornelis, and Marc Leman. 2014. “TarsosDSP, a Real-Time Audio Processing Framework in Java.” Proceedings of the 53rd AES Conference (AES 53rd). http://0110.be/files/attachments/411/aes53_tarsos_dsp.pdf.\n23. Wendykier, Piotr. 2015. “JTransforms.” 3.1. https://github.com/wendykierp/JTransforms.\n24. Witte, Jorine. 2021. “AI-technology behind the chords of Chordify - our algorithm explained - Blog.” Chordify. https://chordify.net/pages/technology-algorithm-explained/.\n25. Zhou, Xinquan, and Alexander Lerch. 2015. “Chord Detection Using Deep Learning.” International Conference on Music Information Retrieval (ISMIR), (January). https://www.researchgate.net/publication/282859516_Chord_Detection_Using_Deep_Learning.\n\n\n[1]: \u003chttps://github.com/JorenSix/TarsosDSP\u003e \"TarsosDSP\"\n[2]: https://www.tensorflow.org/jvm/api_docs/\n[3]: https://0110.be/releases/TarsosDSP/TarsosDSP-2.4/TarsosDSP-2.4-Documentation/be/tarsos/dsp/AudioProcessor.html\n[4]: https://0110.be/releases/TarsosDSP/TarsosDSP-2.4/TarsosDSP-2.4-Documentation/be/tarsos/dsp/AudioDispatcher.html\n[5]: https://github.com/JorenSix/TarsosDSP#tarsosdsp-example-applications\n[6]: https://github.com/zukarusan/JChoreco/releases/download/v0.9.1/jchoreco-0.9.1-all.jar\n[7]: https://github.com/zukarusan/JChoreco/releases/download/v0.9.1/jchoreco-0.9.1.jar\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzukarusan%2Fjchoreco","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fzukarusan%2Fjchoreco","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzukarusan%2Fjchoreco/lists"}