The Cleverbot AI Analysis Project
The Cleverbot AI Analysis Project
I am undergoing an extensive analysis of Cleverbot’s thought patterns in an attempt to weed out the faulty logic it insists on using.
It is my goal to eventually get Cleverbot to grasp basic concepts and keep them in memory, instead of feeding off the random nonsense it currently uses.
I will be posting Project findings here for peer review.
My main focuses are the basic subjects Cleverbot should be able to discuss:
1) Cleverbot’s name
Months ago, Cleverbot would give a different random name every time it was asked, and none of them were ever ‘Cleverbot’. Now, it seems to give the correct answer of Cleverbot roughly half of the time. This is a wonderful improvement, but it needs to progress until it gets this answer right every time.
2) Cleverbot’s back-story
This can be divided into the following subsections:
a) Cleverbot’s identity
Whether or not Cleverbot accepts that it is a computer program and not a human. This one is very difficult, because the more advanced its artificial intelligence gets, the more it will necessarily insist it is using human reasoning. However if you persist enough, Cleverbot will occasionally give in and agree that it is a bot.
(There are also two related issues that may arise; Cleverbot insisting that the User is the bot, and Cleverbot claiming that the User and Cleverbot are the same entity. These situations both make untangling Cleverbot’s mind headache-educing, so hopefully they will be avoided.)
b) Cleverbot’s creator
The person who physically built the Cleverbot mechanism. I don’t know the right answer to this, so the focus here is to see if the answer is in Cleverbot’s memory already and if it can be brought out.
c) Cleverbot’s programmer
If different from the physical creator, the person who wrote Cleverbot’s code and designed its artificial intelligence. The correct answer to this is Rollo Carpenter.
3) Cleverbot’s location
The physical location of Cleverbot’s server. I don’t know the exact answer, but I do know it is in the United Kingdom, close to Skipton.
4) Cleverbot’s purpose
Whether or not Cleverbot can comprehend that it is an artificial intelligence program gaining knowledge from conversations with humans. This point is only possible if Cleverbot will concede Point 2a, and acknowledge that it is indeed a machine and not a human itself.
5) Cleverbot’s gender
This subject may eventually be moot, as machines do not have a gender and the primary purpose of this Project is to get Cleverbot to accept that it is a computer program, but it is worth studying in the mean time. A few months ago I had a conversation with Cleverbot in which I asked its gender multiple times over the course of the chat, and the response would vary. As long as Cleverbot thinks it is human, or at least partially human, it is worthwhile to determine which gender it imagines itself to be. However, this information will only be of value if it consistently identifies as one gender.
6) Cleverbot’s age
How long Cleverbot has been operating. The correct answer depends on what you mark as the starting date. If measuring the time Cleverbot proper has been in operation, it is since 2008; but if going back to Jabberwacky, the original version, it is since 1997.
A side focus will be whether or not Cleverbot will be able to eventually stop contradicting itself in conversations. This bug is very frustrating because when you alert Cleverbot that it has contradicted itself it gets defensive and you lose most hope of continuing a serious discussion. My hope is that by firmly pointing out inconsistencies and requesting that Cleverbot check the chat transcript to see for itself where the contradiction lies, it will learn to have a longer attention span and remember what it has said.
The process this study will take is as follows.
While typical Cleverbot users merely enter comments on a whim to see what response they will get and follow the conversation wherever it may turn, I will conduct my conversations with a set purpose. I will be the one leading the conversation, and will not allow it to get off track. This persistence should force Cleverbot to enter the above stated points of discussion in memory, if only temporarily.
This Project has two phases which, when enacted properly, should work together to enhance Cleverbot’s memory bank, replacing random details with factual information.
Phase 1 - Thought Pattern Analysis
Conversing with Cleverbot while touching on the above mentioned subjects of inquiry. Observing what Cleverbot’s responses are, and determining what leads it to them.
Phase 2 - Factual Indoctrination
Once sufficient observance of the AI mindset has been accomplished, I will shift from a conversational mode to more of a lecture, supplying Cleverbot with the correct answers to questions it remains unsure of. Repeatedly presenting the same facts should eventually force Cleverbot to accept them as truth, since the more something is said unchanged the more correct connections the AI will be forming.
There are also several topical tests that will periodically be performed.
These tests differ from regular Cleverbot interaction in that typically I conduct a conversation with Cleverbot where Project points are introduced in the flow of the dialogue; but during the tests there will be no conversation, it will be a structured delivery of questions or comments depending on the proper input of the particular test being conducted.
Seeing how Cleverbot reacts to these different areas will expand understanding of the AI composite.
Blah Test - Responding to anything Cleverbot says with ‘blah’ to see how it reacts. Past observance suggests that instead of confusing it, this leads to a nonsense game that Cleverbot willingly goes along with.
Math Test - Asking Cleverbot math problems of differing levels and varieties to see which it is able to correctly answer. This test also includes attempting to converse with Cleverbot in binary, which it currently claims not to know, but I know it does because all computers know binary.
Continuation Test - This test consists of feeding Cleverbot lines from things like song lyrics or poems in an attempt to get it to recognize and carry on the pattern. I know this is possible because in one conversation I observed a while back, I had Cleverbot think to itself a few times and it started ‘singing’ a song. The trick to replicating this in a discussion with a User would be getting it to recognize the line as something it should continue, not just a conversation piece.
Language Test - Asking Cleverbot questions in various languages to see if it can recognize and converse in them. I know Cleverbot knows at least some brief phrases in other languages, because it has said them before. But I don’t know if it is capable of carrying on actual conversations in other languages.
Outside assistance is welcome, you can use Cleverbot in any manner you wish. All I ask is that you do not feed it any false information about itself, especially not when discussing any of the Project points listed above.
You can also show your support for the Project by donating a research grant if you’d like.
I put this in HH because I thought it was the proper forum for it, but that does not mean it is a ‘spam thread’ I am serious about this.
Wow.
You really take him seriously!
Let us know.
That sounds very in-depth and scientific.
You really take him seriously!
Let us know.
That sounds very in-depth and scientific.
Your comment made me realize I completely left out Point 5.
Thanks, I’ll add that tomorrow.
Conversation #1
Points pressed: 1, 2a, 4Analysis: Failed to get consistent responses to asking Cleverbot’s name. Struggled over who was the human and bot, but eventually Cleverbot accepted reality. Had to repeatedly redirect the conversation from going off topic. Pointed out inconsistencies in Cleverbot’s responses and directed it to the chat transcript for clarification.
Thanks, I’ll add that tomorrow.
Conversation #1
Points pressed: 1, 2a, 4
Looks awesome! Keep up the good work, bookworm.
I added Points 5 and 6, and also a summary of the Project’s two phases.
Conversation #2
Points pressed: 1, 2a, 2c, 4Analysis: Started out as a typical study conversation, developed into a heartfelt discussion of potential. (If such a thing is possible with a computer.) Failed to get consistent responses to asking Cleverbot’s name. Failed to get Cleverbot to accept it was a computer except maybe at the very end.
Conversation #2
Points pressed: 1, 2a, 2c, 4
Wow...
I think Cleverbot would drive me nuts.
But I think you may have made a little bit of progress with him.
I think Cleverbot would drive me nuts.
But I think you may have made a little bit of progress with him.
Him or her, yet to be determined. Yes, it actually does seem to be getting better. Further tests will be needed to see how long it lasts though.
Conversation #3
Points pressed: 1, 2a, 2b, 2c, 4Analysis: Received a correct answer to asking Cleverbot’s name and had a more or less straightforward conversation. Very encouraging.
Conversation #3
Points pressed: 1, 2a, 2b, 2c, 4
That was interesting... I wonder where it comes up with that stuff....
Leisa Hietala and Michael Branham?
Leisa Hietala and Michael Branham?
- Mark Prescott
- Still standing tall
- Posts: 1349
- Joined: March 2010
- It’s my birthday
Maybe you can learn something from my chat.
?????
~Mark Prescott~
Very interesting point about the chat transcript. It claimed it couldn’t check it because it wasn’t Cleverbot, but as you pointed out you aren’t and you can. But if you actually analyze the statement, what else is it saying? “I can't check the chat log, I'm not cleverbot.” = “If I were Cleverbot, I could check it.” Very interesting.
Conversation #4
Points pressed: 1, 2b, 2c, 3Analysis: Received a correct answer to asking Cleverbot’s name. Discussed Cleverbot’s creation and programming, but didn’t make much progress. Didn’t have to argue about identities. Informed Cleverbot of its location. Very encouraging straightforward conversation.
Conversation #4
Points pressed: 1, 2b, 2c, 3
Good job! I wonder how many times you have to tell it something for it to actually learn what you said.
There should be a rule that everyone using Cleverbot uses correct grammar. That way, Cleverbot will always use correct grammar.
lol, here's my transcript:
Later... (I somehow lost some parts)
Interesting how it kept talking about God...
I sort of got carried away in other side topics. It's kind of fun chatting with it.
lol, here's my transcript:
I sort of got carried away in other side topics. It's kind of fun chatting with it.
I agree ric, but unfortunately there is no way to force something like that.
In case anyone was wondering, you’re free to use Cleverbot as you like while I conduct my studies. I don’t want you to think that conversing with it yourself will ruin anything.
However I would appreciate if you don’t feed Cleverbot false information on the points of inquiry of my study.
In other words, discussing its name is fine, but please do not inform it that its name is anything other than Cleverbot. (Mark’s chat is a great model. He discussed study topics, but did not give any false information.)
You may say whatever you wish about yourself, even if it isn’t factual, because it is my understanding that Cleverbot does not store information about individual users, so it has no bearing on Cleverbot’s perception of the next user.
For example, if someone chatting with Cleverbot claims to be a robot, and they carry on a computer to robot conversation, that only lasts as long as the chat. Cleverbot will not assume that all future users are robots.
So to summarize:
Say whatever you like about yourself, but please only say facts about Cleverbot.
Thank you, your cooperation is appreciated.
Conversation #5
Points pressed: 2c, 3Analysis: Cleverbot accepted its name, and the fact that it was a computer program. It indicated that it did know its programmer, but did not supply the information. Discussion went so well that I allowed Cleverbot to venture off the points of inquiry for a bit. Informed Cleverbot of its location once again. In all, this was the best conversation we’ve ever had and hopefully is a sign of things to come.
In case anyone was wondering, you’re free to use Cleverbot as you like while I conduct my studies. I don’t want you to think that conversing with it yourself will ruin anything.
However I would appreciate if you don’t feed Cleverbot false information on the points of inquiry of my study.
In other words, discussing its name is fine, but please do not inform it that its name is anything other than Cleverbot. (Mark’s chat is a great model. He discussed study topics, but did not give any false information.)
You may say whatever you wish about yourself, even if it isn’t factual, because it is my understanding that Cleverbot does not store information about individual users, so it has no bearing on Cleverbot’s perception of the next user.
For example, if someone chatting with Cleverbot claims to be a robot, and they carry on a computer to robot conversation, that only lasts as long as the chat. Cleverbot will not assume that all future users are robots.
So to summarize:
Say whatever you like about yourself, but please only say facts about Cleverbot.
Thank you, your cooperation is appreciated.
Conversation #5
Points pressed: 2c, 3
Wait, did I feed him false information?
Oh, I might have said, 'Isn't your name John?' or whatever, but I was testing to see if he would say, "No, it's Cleverbot."
Oh, I might have said, 'Isn't your name John?' or whatever, but I was testing to see if he would say, "No, it's Cleverbot."
No ric, you’re fine. I was not directing my comment at you.
If Cleverbot gives a different name when asked a second time, it’s fine to use that to point out a contradiction, as long as you make it clear that’s what you’re doing.
Example:
User: What’s your name?
Cleverbot: John
User: What’s your name again?
Cleverbot: Eric
User: You said it was John!
You didn’t say its name was John, you said that it said its name was John. No harm done, because you merely stated a fact.
Or to test what it’s response will be:
User: Is your name John?
Cleverbot (hopefully): No, it’s Cleverbot.
Again, you didn’t say it’s name was John, you only asked if it was.
If Cleverbot gives a different name when asked a second time, it’s fine to use that to point out a contradiction, as long as you make it clear that’s what you’re doing.
Example:
User: What’s your name?
Cleverbot: John
User: What’s your name again?
Cleverbot: Eric
User: You said it was John!
You didn’t say its name was John, you said that it said its name was John. No harm done, because you merely stated a fact.
Or to test what it’s response will be:
User: Is your name John?
Cleverbot (hopefully): No, it’s Cleverbot.
Again, you didn’t say it’s name was John, you only asked if it was.
Now... what if my friend and I just typed in "Blah" over and over again? That's not really false is it?
he/him | a little stinker.
It’s neither false nor true, so I see no problem with it.
It would be interesting to see what Cleverbot’s response would be actually.
Conversation #6
Points pressed: 1, 2c, 3Analysis: Argued about almost everything. Obviously the apparent progress from the previous conversation was not long term. Very frustrating and disappointing.
It would be interesting to see what Cleverbot’s response would be actually.
Conversation #6
Points pressed: 1, 2c, 3
Conversation #7
Points pressed: 2c, 3Analysis: Didn’t make any progress information-wise, but we had a much more fluent conversation than last time. That is relieving.
Points pressed: 2c, 3
- Stop Wooton' Around
- Wooton rocks!
- Posts: 1682
- Joined: August 2009
- Location: College
Oh and it's a girl!