Trust & Safety teams face a constant barrage of new challenges, from emerging threats to evolving inappropriate behaviors. Platforms have a multi-layered structure to protect users, from AI and ML models for automated detection to review queues, where internal moderation teams and external BPO agents provide a critical layer of human supervision and moderation decisions. Queues are the operational core for this layer of review. This layer can be slowed by bottlenecks that affect triage and average handle time, or require engineering tickets to make any changes. We built Queues in the Cinder platform to be flexible and configurable to give Trust & Safety teams the power to respond swiftly to challenges. This post will look at what powers behind this operational configurability. We’ll then look at how this flexibility enables faster process change, supports more automated reviews, and produces high-quality labeled data for engineering teams.

Example in Practice: Operationalizing Queues

To illustrate the application of Queues in Cinder, imagine a platform in growth mode that is quickly adding geographies and product lines. As the platform adds new geographies, it will take into account new cultural and regulatory dynamics and adjust policies to reflect this expansion. Further, it may need to add language capabilities for human reviewers. Likewise, as a new product line launches, new policies, content review renderers, and moderator instructions may be required.

Using Queues, it is possible for the platform’s Trust & Safety team to create new queues or adjust current queues by simply configuring the UI. Once the team has a plan in place to tackle its new requirements, setting up the queues and onboarding reviewers to them takes minutes, not months. Let's look at how this functionality works in Cinder.

Simplifying Process Changes

Centralizing user and detection data in the Cinder platform means Trust & Safety teams can overcome process change hurdles that prevent timely action. We wanted to make queue management responsive, so Ops managers could take action immediately in Cinder, without waiting on engineering teams. The Queues function empowers admins to directly create or modify queues. This autonomy significantly reduces turnaround time for implementing new strategies or responding to emerging risks.

‍

‍

Identifying and prioritizing critical issues can be like finding a needle in a haystack. The flexibility of Queues helps teams effectively categorize and route content based on predefined criteria, ensuring that high-priority or sensitive content is quickly brought to the attention of the right agents. Automated workflow rules can do a lot of the heavy lifting for triage, routing, and routine decisions. These features were designed to help Trust & Safety leaders gain control over the operational processes crucial to reducing potential harm on platforms.

For decisions where accuracy is paramount, the Multi-Review Settings in Queues add an extra layer of scrutiny. This feature is particularly beneficial in scenarios where the consequences of erroneous labels are significant. Requiring multiple reviews and consensus before finalizing a decision greatly reduces the risk of mistakes.

Policy alignment is another area we sought to improve over traditional processes. In Queues, the integration of platform policies into the decision-making process is seamless. The Policies tab in Queue Settings and the decision options in the Action Panel ensure that every action taken by an agent is in line with the organization's guidelines and values. This alignment is crucial in maintaining consistent governance and creating high-quality labeled data for model training.

Queues is not intended to be just a feature. It’s a strategic asset for any Trust & Safety team. The highly configurable and customizable nature of Queues empowers teams to get out of reactive mode, and proactively mitigate harm.

Queue Settings

The strength of Queues is flexibility. Teams can finely tune this core Cinder function to meet their unique Trust & Safety needs. Each setting is designed with the intent of providing flexibility, precision, and control for review management. Here, we’ll look at some of the most important settings:

‍Target Handling Time: This setting allows admins to define a target time frame for resolving jobs within a specific queue. By setting these benchmarks, managers can ensure timely responses, crucial for maintaining the quality and efficiency of operations. This time-bound approach also helps in managing workload and setting clear expectations for agents.

‍Single or Multiple Policy Selection: By enabling this feature, admins can determine whether agents are limited to selecting a single policy or multiple-policies per job. This flexibility helps teams streamline their particular decision-making processes and maintain clarity in policy application.

‍Change Queue Option: The ability for agents to move jobs between queues offers flexibility in handling misrouted content. This feature ensures that each job is reviewed in the most appropriate context, enhancing the efficiency and accuracy of the review process.

‍Escalate Jobs Capability: Enabling agents to escalate jobs to a higher-tier queue ensures that more complex or sensitive issues receive the appropriate level of scrutiny and decision-making expertise. This hierarchical approach to content review fosters a more nuanced and effective handling of challenging cases.

‍Flaggable Field Guided Review: This innovative feature allows agents to make policy selections on every attribute of an entity deemed "flaggable." It's a critical tool that allows for simultaneously moderating content, analyzing in detail, and rich data labeling for AI training, enhancing overall model quality and accuracy.

Safety Tools: Recognizing the sensitive nature of some content, admins have the option to set images and videos blurred and displayed in grayscale by default. This consideration not only safeguards the well-being of agents but also aligns with broader organizational standards for handling explicit or sensitive material, including specialized settings for handling CSAM.

‍User Reports Visibility: This setting gives admins the flexibility to control what agents can view with respect to the reports that led to a job being enqueued. You can choose to show or not show reports, depending on the context you want to provide.