Conversation Tests

Conversation tests are currently in invite-only beta, and will be available under Platform and Enterprise plans when launched.

As your bot becomes increasingly fluent, it becomes more and more important to ensure that your new content doesn't interfere with the old. While testing your changes with Dexter's phone tester helps to ensure that things are working as you're writing, you'll eventually find yourself with more content than you can test in one sitting.

Conversation tests are the long-term testing tool of choice. These are scripted conversations you can have with your bot at the click of a button. Starting from "hello", each test lets you check every assumption you've made about how your discussions will unfold. Over time you can build up a library of conversation tests that covers your entire bot, helping to ensure that new changes haven't disrupted old behavior.

Important Facts

Writing your first test

The test page

You'll start with the clipboard icon in the toolbar, where you'll be prompted to create a new test. When you do, you'll come to the test editor:

A new test

This is where you'll script out your conversation. Each line has you enter something a user would say, then fill out what you expect the bot's response to be. If your bot is simple, you might be able to populate a whole conversation off the top of your head:

Know-it-all

For most bots, you'll probably have an idea about what prompts will work, but you might not know the actual response. In that case, go ahead and fill in what you can, run the half-finished test, and use the correct answers to populate the correct responses:

Know-some-of-it

Pro Tip:
Give your new test a good name before you finish - something that you'll recognize weeks from now, and that will keep it distinct from the rest of your tests.

Organizing tests

How you organize your tests will largely depend on the nature of your bot. If you're writing a choose-your-own-adventure game, you could organize your tests by room and player state:

Test by room

If you're building an FAQ, you could group questions by theme:

Test by theme

A quiz could easily be organized by question:

Quiz test

What's important is that the tests are organized and named in a similar fashion to your bot - this will make adding new tests easier and make it more straightforward to keep the tests you've already made up to date.

Tips & Tricks

Don't be afraid of helpers

Having a few testing utilities built into your bot can be useful during development and for writing tests. For example, if your bot is highly variable dependent, you might want to add some secret triggers that let you jump to a particular state:

+ set-state-number-correct *
- <set answers_correct=<star>> OK
 
+ results
* <get answers_correct> == 3 => Perfect score, nice job!
* <get answers_correct> == 2 => You only missed one, not bad.
* <get answers_correct> == 1 => Well, you got one right, at least.
* <get answers_correct> == 0 => Uhm...were we taking the same test? Big 0, I'm afraid.
- Hmmm...that's weird.

Helpers