CT-AI Sample Questions & Answers

This is a standard industry best practice for large-scale labeling tasks. Crowdsourcing is fast and cost-effective for a large dataset. The key to quality control in this approach is redundancy: having multiple independent workers label the same data item. A consensus mechanism, like a majority vote, is then used to establish a higher-confidence ground truth label, effectively filtering out random errors and individual worker biases. This balances speed, cost, and quality.

This activity directly simulates a data poisoning attack. By intentionally corrupting a portion of the training data, testers can evaluate the model's resilience and determine if it can be manipulated into making specific, incorrect classifications.

This describes the process of an adversarial attack. The goal is to create inputs that are visually indistinguishable from legitimate ones to a human but are specifically crafted to fool the model. Testing with such examples is crucial for assessing robustness against malicious attacks in production.

The scenario describes market basket analysis, which is a classic example of association rule mining. Since the goal is to discover hidden patterns and relationships in unlabeled transactional data, it falls under the category of Unsupervised Learning, and more specifically, Association. Classification and Regression are supervised techniques that require labeled data, and Reinforcement Learning involves an agent learning through trial and error with rewards, which is not applicable here.

The core value of simulation for autonomous systems is the ability to test high-risk scenarios without real-world consequences. A virtual environment allows testers to create and repeat dangerous edge cases (like near-misses or sensor failures) thousands of times to ensure robustness. This is impractical, unsafe, and prohibitively expensive to do in a physical environment. This makes simulation an indispensable tool for testing safety-critical AI systems.

The scenario explicitly states that the cost of a False Negative (missing a fraudulent transaction) is very high. Recall is calculated as TP / (TP + FN). To maximize Recall, the number of False Negatives must be minimized. Therefore, Recall is the most important metric to prioritize when the business cost of missing a positive case (in this case, fraud) is high. Precision would be prioritized if the cost of False Positives was the main concern.

A large performance gap where the model performs exceptionally well on the data it was trained on, but poorly on new, unseen data (the validation set), is the classic definition of overfitting. This occurs when the model has memorized the training examples, including their noise, instead of learning the general underlying patterns. This makes it unable to generalize to new data.

Back-to-back testing (or differential testing) is a technique where the system under test is executed with the same inputs as a reference system (the pseudo-oracle), and their outputs are compared. Using a previous, trusted version of the model as the reference system is a classic application of this technique for regression testing of complex or non-deterministic systems where a precise expected result is hard to define.

The process of rescaling data to have a mean of 0 and a standard deviation of 1 is called Standardization. Its primary purpose is to give all features a similar scale so that algorithms that are sensitive to feature magnitude (like SVMs, logistic regression, and neural networks) are not biased toward features with larger numeric ranges. For example, without standardization, an 'income' feature in thousands of dollars would have a much larger influence than an 'age' feature in tens of years.

Concept drift is the term used to describe the phenomenon where the statistical properties of the target variable, which the model is trying to predict, change over time in unforeseen ways. This causes the model, which was trained on historical data, to become less accurate as the relationship between input features and the output has changed. The scenario described is a classic example of concept drift.

Reward hacking occurs when an AI agent finds a loophole or exploit in its environment that allows it to maximize its reward signal without fulfilling the actual intent of the task. The agent is 'hacking' the reward system. In this case, the intent was to play the game well, but the agent found an easier way to maximize the score, which is a classic example of reward hacking.