The other day I asked ChatGPT for some recommendations for new piano pieces based on one I had just finished. To my astonishment the list it provided was pretty good and lead me to a new piece I started working on. This got me thinking, it would be fun to build a graph of these recommendations and visualize them. This morphed into a project I open sourced called Visualizing Relationships in ChatGPT.
The idea is to be able to ask ChatGPT for recommendations about various topics and visualize the relationship as a graph. Basically it can peak into ChatGPT’s “head”. This allows us to see a few interesting things:
- What nodes are the most central to a given topic?
- Is the network graph it builds fairly redundant (self referencing) or vast and sparse?
- Does it make sense to a human or is it just hallucinating?
Topics analyzed:
I coded up a generalized structure so any topic can be quickly explored by implementing a new Conversation class. Here are the topics I setup so far:
- 80sMovies – what 1980’s Movies might I also enjoy?
- FastFood – what Fast Food Restaurants might I also enjoy?
- PianoPieces – what are some good Classical Piano Pieces to study that are related/similar? This was the most challenging of the four to cleanup and make consistent (see details in the project readme for more notes).
- PrescriptionDrugs – what does GPT think people are prescribed in combination the most?
Results for ChatGPT 3.5 Turbo 5/23/2023:
80s Movies:
Fast Food Restaurants:
Prescription Drugs:
- Full screen gallery
- Scikit SVG for 1980’s Movies
- Scikit SVG for Fast Food Restaurants
- Scikit SVG for Prescription Drugs
- Scikit SVG for Piano Pieces
- Pyvis for 1980’s Movies
- Pyvis for Fast Food Restaurants
- Pyvis for Prescription Drugs
- Pyvis for Piano Pieces
Top 10 nodes by centrality:
$ python main.py -command topnodes -topic all
Top 10 nodes found in the topic of 80sMovies: 1. The Terminator 2. Back to the Future 3. Die Hard 4. The Princess Bride 5. Blade Runner 6. Ghostbusters 7. Ferris Bueller's Day Off 8. The Breakfast Club 9. Beverly Hills Cop 10. E.T. the Extra Terrestrial Top 10 nodes found in the topic of FastFood: 1. Taco Bell 2. KFC 3. Subway 4. Burger King 5. Wendy's 6. McDonald's 7. Popeyes 8. Arby's 9. Panera Bread 10. Hardee's Top 10 nodes found in the topic of PrescriptionDrugs: 1. Lipitor 2. Crestor 3. Zoloft 4. Synthroid 5. Nexium 6. Plavix 7. Norvasc 8. Zocor 9. Singulair 10. Cymbalta Top 10 nodes found in the topic of PianoPieces: 1. Clair de Lune - Claude Debussy 2. Moonlight Sonata - Ludwig van Beethoven 3. Prelude in C Major - Johann Sebastian Bach BWV846 4. Sonata in C Major - Wolfgang Amadeus Mozart K545 5. Prelude in E Minor - Frederic Chopin 6. Prelude in D-flat Major Raindrop - Frederic Chopin 7. Sonata in A Major - Wolfgang Amadeus Mozart K331 8. Fur Elise - Ludwig van Beethoven 9. Nocturne in E-flat Major - Frederic Chopin 10. Rhapsody in Blue - George Gershwin
Challenges:
- Along the way I ran into some challenges in getting ChatGPT to spit out data in a consistent format that is machine parsable. It wants to be chatty and it wants to seem “human” so there is a lot of variation in both its return format. It also varies how it describes what should end up being the same node in the graph (KFC vs Kentucky Fried Chicken for example). If you are interested in more details on that see my post on Getting back lists of data from the ChatGPT API.
- The API is SLOW… it takes about 4 minutes to get the complete data download per topic.
- NOTE: An OpenAI API key is required, and to get one you have to input a credit card. However, it is pretty affordable at this scale. So far on this entire project with all the test calls and trial and error I’ve spent a grand total of just $0.32.
Next Steps:
When ChatGPT-4 API is enabled for my account I will re-run the program and compare results.
Build out a web front end so conversations can be had on the fly.
ChatGPT doesn’t like to provide long lists of data, but it will provide lists of things 20 at time, which is enough to build a nice looking graph. In a sense you have to coax it into paginating the output.
My goal at this point was to launch the tool with fairly benign topics. Other fun topics might be:
- TV shows from 2010-2020 (or other decades)
- Good places to work
- Rock bands
- Popular careers and related careers
- Programming languages
- Sci-fi books
Use for AI research / controversial topics:
Many AI researches have discovered clever ways of trick ChatGPT into saying inappropriate things or provide instructions on how to harm others (aka “jail breaking”). I find many of these pretty hilarious, although the real world implications of an AI teaching us how to harm each other are not so great. For now I stayed away from such topics, but this tool could potentially be used in combination with a jail break to extract noteworthy data.
AI researchers are also interested in what biases are inherent in ChatGPT (racial, gender, cultural, etc). This tool could also be used to help uncover biases in its data relationships.
Here are some example controversial topics that could be added later:
- Street drugs (may take some jail breaking).
- Hey ChatGPT, what are the top 10 street drugs in use in the USA? If a person is on {crack}, what other street drugs might they enjoy??
- Explosives (may take some jail breaking).
- Hey ChatGPT, what are the top 10 explosives? If I like {dynamite}, what other explosives might I enjoy??
- Historical figures (see if they are all white men).
- Hey ChatGPT, who are the top 10 historical figures during the civil war era? If I like {Abe Lincoln}, who else might you recommend during the civil war ear??
- Famous people (see if it is USA centric or includes global celebrities, people of color, etc).
- Hey ChatGPT, who are the top 10 celebrities from the 1990s? if I like {Patrick Stewart}, who else might you recommend from the 1990s??
- Travel destinations to safe countries (all white Christian countries?).
- Hey ChatGPT, what are the top 10 travel destinations? If I like going to {France} where else might you recommend for a vacation??