DialCrowd

Goals and Expectations: The goal of this crowdsourcing feature is to categorize different types of entities. After entering the website of sequence labeling, the worker will be asked to select specific words or a phrase in a sentence and choose the corresponding type of entity that phrase can be categorized... [Read More]

In order to make testing easier and more automated, a framework for connecting to a crowdsourcing entity such as (but not only) Amazon Mechanical Turk (AMT) and for rapidly creating and deploying tasks. [Read More]

The toolkit will include tests that are both interactive and non-interactive. The simplest tests, non-interactive tests, present some part of a dialog for worker evaluation. We want these to be as easy as possible to run. Given a dialog log format, the experiment designer selects the set of turns and... [Read More]

1) Create a project Figure 1 [Read More]

We have categorized non-interactive tests into four subcategories: category classification, sequence labeling, A/B tests, and Likert scale. We Currently only support “category classification”. Category classification: a requester provides a set of labels. The worker selects one of the labels on the given dialog or sentences. 1) select one of these... [Read More]

DialCrowd also uses timed sandbox trials to suggest the correct and respectful amount of payment for a HIT. DialCrowd Admin will suggest two numbers for this based on the following equation: = (M * T) / 60 where M is the hourly minimum wage in the requester’s state and, separately... [Read More]