A.P. Wage History
Top Worker Review
Yoyo Average Pace
Answer a survey about your opinions about chatbot - $0.40
Unrated
Unrated
Rejected (All)
$3.65 / hour
00:06:35 / completion time
- HIT Rejected
Pros
I'm still here and kicking.Cons
Firstly, everything is mirrored on a digital smartphone to the right of the screen, which takes up about 25% of the screen. This is because the survey is not actually active. It's set to PREVIEW MODE, which allows Workers to place bookmarks in the study, create QR codes to it, view hidden questions, ignore validations, and restart the survey at any given time. This also includes turning off mobile viewing, which is set on by default.The second negative is that there's a broken counter on the bottom of the page that shouldn't be there, which tells the Requestor how many times you clicked on the page and how long you spent on it. This is obviously not supposed to be visible to Workers, but they left it in without realizing it, because again, they published the study without actually completing it. Even still, the counter is broken because it does not count when you click on the audio files you're supposed to listen to, or when you click the button to go to the next page.
This means that it will indicate to the Requestor that you spent 0.00 seconds on each page, and clicked it zero times. Obviously this is impossible, but given how incompetently this study is put together, I wouldn't be surprised if the Requestor truly believed people were somehow skipping through the end of the study under zero seconds.
As for the actual study itself, you'll first have to sign your name digitally. Always a good sign. Then the actual first page requires you to listen to several audio files of a chatbot, then read the response by a human, then listen to the audio record of the chatbot responding to them, and so on. You do this again on the second page. On the third page you confirm you read everything and listened to everything that was said. You're then supposed to write about what was discussed.
I wrote down that the two discussed clothing returns and other clothing options, which they did. But I didn't go in detail, as there are 13 responses, with some being almost 25 seconds long. It's honestly a lot to keep track of, but I imagine that if you don't go in depth they'll reject you.
You then get a page of 9 questions which are all the basic "Strongly agree" to "Strongly disagree" about topics like "This chatbot easily got affected by the customer's emotions."
Naturally I disagreed with most of these, because the chatbot showed no emotion. It's a BOT.
This is followed by another 6 questions on the next page, such as "This chatbot understands my problem.", which are increasingly odd, because somehow I'm judging the chatbot as if it were talking to me, rather than the other person in the conversation. Apparently the Requestor forgot who was talking to whom in this scenario.
On the page after this, you'll encounter 4 more questions, such as "When I listened to the customer interaction with the chatbot, I" to which you can choose answers ranging from "Do NOT concentrate fully" to "concentrate fully",
and
"When I listened to the customer interaction with the chatbot, I:" to which you can answer "am NOT absorbed intently" to "am absorbed intently".
It's insane.
The next page gets even more odd. It starts by asking "I often ask the chatbot to talk to me out of curiosity." to which you can select "Strongly disagree" to "Strongly agree". Again, I never spoke with the chatbot.
There are 6 of these questions in total, such as "When I hear a new thing from the chatbot, I'm eager to talk to the chatbot."
The next page includes EIGHT more questions, "When I hear a new thing from the chatbot, I'm eager to talk to the chatbot." and "I often ask the chatbot to talk to me out of curiosity."
Yes. That's right. Every single question repeats again, all eight.
On the next page you get 3 more questions, "I intend to accomplish my tasks by using this chatbot." followed by 3 more questions on the next page like "Being a user of this chatbot has been a satisfying experience."
Again. I have not used the chatbot.
The next page has TWENTY questions! These range from "Talking to the chatbot challenges me to perform to the best of my ability." to which I answer how much I agree or disagree to things like "I feel during interaction with the chatbot." to which I answer how engrossed I am.
Other answers include things like whether I feel "Fully influential" to "Fully influenced", or "Fully dominate" to "Fully submissive", or "Fully autonomous" to "Fully guided". This of course includes in-between points like "Somewhat autonomous" or "Guided".
What.
After answering all of those questions, there's TEN more on the next page. These include questions such as When I hear a new thing from the chatbot, I'm eager to talk to the chatbot." and "I often ask the chatbot to talk to me out of curiosity."
Yes. The same questions for a THIRD TIME!
Finally, you get to a page with only five questions, and these ones are more standard, like "I get frustrated if others are not available when I need them".
Five more questions of the same nature on the next page.
Then it's SIXTEEN more questions! Like "I embrace changes to who I am." which don't mean anything at this point.
Then another EIGHT questions. These ones make even less sense somehow; questions like "I worry that [firm] will abandon me as a customer" and "It helps to turn to [firm] in times of need".
What firm? Why would I turn to them? Do I work for them? Does the chatbot? WHAT!
Finally, you come to a page with only four questions. It's just standard demographic stuff. Age, income, that sort of thing.
Then you get your completion code, except there's also a blank box at the bottom for you to write something in? No instructions on what to do.
Guess what! It's not over!
Now FIVE more questions! These include things like "I tend to want others to admire me." which make no sense coming after the survey has already ended, but nothing in this study makes any sense, so why not?
After you answer them all, it's finally over. For real.
And A.P. will thank you with a big sweet rejection note saying "You did not follow the instructions so your responses were removed. They do not help us."
I have no idea what they want from us.
I have dealt with insidious Requestors in the past, who try to steal free work from Workers and reject to avoid paying. I've dealt with incompetent Requestors who have no idea how to use Mturk and reject inappropriately. I've even dealt with Requestors who reject arbitrarily, and you'll never have any idea of why your work sometimes makes it through and other times doesn't.
This is the only time I've ever questioned whether or not the Requestor I was working for was sane. The questions boardline on incoherent. Their feedback is nonsensical. They expect you to answer 104 questions for less than half a dollar. They're crazy.
Edit: Received a response back about the rejection. I asked exactly what I did wrong and the answer was "The descriptive question that asked participants to describe what was the script does not match the content discussed in the survey. Hope that helps. Best."
Amazing. So yes, it seems that if you don't practically transcribe every line of dialogue, you'll be rejected. Summing it up isn't enough.
Advice to Requester
Scrap your entire study. It's broken from top to bottom. There's no way you'll get any valid data from it. I'm serious.Try again and do better. And next time, given actual decent feedback. If you feel a Worker has done something wrong, explain what they did, and how. Don't just brush them off and blame them for not helping YOU when everything about your study gets in the way of them helping YOU.
Jul 13, 2023 | 1 worker found this helpful.