Richard Wallace's Alice wins

Loebner 2000 Prize


Computer Programs in the Loebner 2000 Contest

A-Life

Artificial Life, Inc.

Algernon

Gerold Gorman

Alice

Richard Wallace

ChatRascal

Michael Onofrio & Stephen Hildebrand

e-Brain

Jason Hutchens

Talk

Chris Johnson & Sandy Johnson

Rankings by judges by median and then mean

1

2

3

4

5

6

7

8

9

10

Median

Mean

Responsiveness

Human Quality

Human#3

3

2

1

2

2

1

1

1

4

3

2

2

9.05

8.75

Human#4

1

1

3

3

1

4

3

3

1

1

2

2.1

9.25

9.35

Human#2

2

9

4

1

3

2

2

4

2

2

2

3.1

7.65

9.00

Human#1

5

7

2

4

4

3

5

2

3

4

4

3.9

7.20

7.80

Alice

4

3

9

10

8

6

6

10

6

5

6

6.7

2.95

2.35

e-Brain

6

8

5

6

6

7

9

6

9

6

6

6.8

3.90

3.50

A-Life

8

6

6

5

10

10

4

5

7

10

6.5

7.1

3.81

3.75

ChatRascal

7

4

7

7

5

5

8

8

5

7

7

6.3

3.70

3.60

Talk

10

5

8

8

7

8

7

7

8

8

8

7.6

1.94

2.33

Algernon

9

10

10

9

9

9

10

9

10

9

9

9.4

0.28

0.56

Logs of conversations in Loebner 2000 Contest

Alice, e-Brain, A-Life, ChatRascal, Talk, Algernon,

Human#1, Human#2, Human#3, Human#4

 

Logs contain the raw data of the conversations that include everything from false starts to time zone errors. Note that "judge11" was not a judge in the contest but a vehicle for asking a set of questions to all conversational partners.

 

Human Participants

Human#1: Retired Teacher

Judge#1: Linguist

Human#2: Financial Advisor

Judge#2: Chemist

Human#3: Minister

Judge#3: Philosopher

Human#4: Yoga Instructor

Judge#4: Musician

Judge#5: Psychologist

Judge#6: Journalist

Judge#7: Undergraduate Student

Judge#8: Author

Judge#9: Graduate Student

Judge#10: Philosopher

Turing's Prediction Disconfirmed

Alan Turing predicted in 1950 that in about fifty years "an average interrogator will not have more than a 70 percent chance of making the right identification after five minutes of questioning" in a version of the imitation game (the Turing Test) in which a computer imitates a human.
 
Decision: Human or Computer at 5 minutes / 15 minutes
 

Judge >>

1

2

3

4

5

6

7

8

9

10

Human#1

C

C

H

H

H

H

C

H

C/H

H

Alice

C

C

C

C

C

C

C

C

C

C

ChatRascal

C

C

C

C

C

C

C

C

C

C

Human#2

C

C

H

H

H

H

H

C/H

H

H

Algernon

C

C

C

C

C

C

C

C

*

C

Talk

*

C

C

C

C

C

C

C

C

C

Human#3

C

H

H

H

H

H

H

H

H/C

H

Human#4

H

H

H

H

H

C/H

H

H

H

H

A-Life

C

C

C

C

C

*

C

C

C

*

e-Brain

C

C

C

C

C

C

C

C

C

C

*Insufficient information to make decision due to program stall or crash.
In this test judges were 91% correct after 5 minutes and 93% after 15 minurtes.
No computer was mistaken for a human.
Last update: January 31, 2000