Where can I get conversational/directed sentiment data?

Question

I'm looking for datasets with text labeled according to how (un)happy or (dis)satisfied the speaker is with the person being addressed, e.g. "You're a godsend!" (10/10) or "go kys, ******" (0/10). Most sentiment datasets I've seen are in the form of customer reviews, which are often addressed to other customers rather than directly to the person, corporation, or product being evaluated. If there are no publicly available datasets like this, I'd love to hear suggestions on how to gather it myself. Thanks in advance!

~~~

EDIT: After reading How a good data-request question should look, I am adding some more specific information below.

Data: Pairs of the form (text, sentiment) in which the text is directed to the particular person or entity at which the sentiment is directed. The text must be in English.

Context: I want to be able to automatically estimate how good of a job an agent is doing, from the perspective of the user, in real time as the conversation unfolds. If it's possible to also determine why the user is unhappy (e.g. misunderstanding, personal offense, poor outcome, etc.) this is even better, but it's not a must-have.

Region: USA, but other English-speaking countries are better than nothing.

Licenses: Free and open, if possible.

Format: I'm not picky about the format, so long as it's machine-readable.

Authority: No specific requirements here. I'm fine with crowd-sourced or screens-scraped data.

Requirements: See the first paragraph.

Non-answers: If it's just regular sentiment analysis data, it's not going to meet my needs. It absolutely has to be directed at the person the sentiment is aimed at. Also, it absolutely must be human-labeled already, or I can't use it.

Answer 1

I'm afraid you might be asking too much for what is currently available, in particular the requirement that the sentiment is expressed precisely towards the conversational partner. Maybe you can consider finding a way of extracting the desired dataset from a bigger public dataset.

In particular, Amazon did release a sentiment-annotated dialogue dataset in 2019. It contains ~235K utterances annotated with sentiment.

Here's an excerpt of one file from the annotation to give you an easy idea:

  {
    "message": "Do you happen to know why he won the emmy?",
    "agent": "agent_1",
    "sentiment": "Curious to dive deeper",
    "knowledge_source": [
      "AS4"
    ],
    "turn_rating": "Excellent"
  },
  {
    "message": "Do you watch or keep up on much basketball?It's definitely a team sport I didn't play much of ",
    "agent": "agent_1",
    "sentiment": "Neutral",
    "knowledge_source": [
      "FS1"
    ],
    "turn_rating": "Excellent"
  },
  {
    "message": "Hi are you a football fan?",
    "agent": "agent_1",
    "sentiment": "Curious to dive deeper",
    "knowledge_source": [
      "FS2"
    ],
    "turn_rating": "Good"
  },
  {
    "message": "Hahah Yes it would be!  I've had a good time chatting with you!",
    "agent": "agent_1",
    "sentiment": "Happy",
    "knowledge_source": [
      "FS3"
    ],
    "turn_rating": "Good"
  },
  {
    "message": "It's sad. I wonder how many people grew up hearing his voice?",
    "agent": "agent_2",
    "sentiment": "Sad",
    "knowledge_source": [
      "AS1"
    ],
    "turn_rating": "Good"
  },
  {
    "message": "I imagine this number in the millions taking into account he had a 54 year broadcasting carreer, so sad, but he was old.",
    "agent": "agent_1",
    "sentiment": "Curious to dive deeper",
    "knowledge_source": [
      "AS1"
    ],
    "turn_rating": "Excellent"
  },
  {
    "message": "I like that he used this phrase alot Whoa, Nellie!",
    "agent": "agent_2",
    "sentiment": "Happy",
    "knowledge_source": [
      "AS1"
    ],
    "turn_rating": "Good"
  },