Goals and Expectations: The goal of this crowdsourcing feature is to categorize different types of entities. After entering the website of sequence labeling, the worker will be asked to select specific words or a phrase in a sentence and choose the corresponding type of entity that phrase can be categorized...
[Read More]
Introduction
What is DialCrowd?
In order to make testing easier and more automated, a framework for connecting to a crowdsourcing entity such as (but not only) Amazon Mechanical Turk (AMT) and for rapidly creating and deploying tasks.
[Read More]
Interactive vs. Non-interactive tests
What template should I use?
The toolkit will include tests that are both interactive and non-interactive. The simplest tests, non-interactive tests, present some part of a dialog for worker evaluation. We want these to be as easy as possible to run. Given a dialog log format, the experiment designer selects the set of turns and...
[Read More]
Interactive tests
How to setup Interactive tests in DialCrowd Admin
1) Create a project
Figure 1
[Read More]
Non-interactive tests
How to setup non-interactive tests in DialCrowd Admin
We have categorized non-interactive tests into four subcategories: category classification, sequence labeling, A/B tests, and Likert scale. We Currently only support “category classification”. Category classification: a requester provides a set of labels. The worker selects one of the labels on the given dialog or sentences. 1) select one of these...
[Read More]
Payment Suggestion
How much do you need to pay to workers?
DialCrowd also uses timed sandbox trials to suggest the correct and respectful amount of payment for a HIT. DialCrowd Admin will suggest two numbers for this based on the following equation: = (M * T) / 60 where M is the hourly minimum wage in the requester’s state and, separately...
[Read More]