Ethics, bias and confidentiality

Know exactly what you send, what generated code is worth legally, and how model biases steer your technical decisions.

10 min · 3 questions

Every prompt is a transmission of data to a third party. Before using an assistant in a professional context, three questions must have a documented answer. Retention: how long does the provider keep your conversations, and who can access them (support, security, court order)? Training: is your data used to train future models, and is the opt-out enabled on your account - consumer plans and enterprise plans almost always differ on this point? Contract: do your employer or your clients allow you to transmit this code, knowing that the confidentiality clauses you signed also apply to a prompt? A code snippet may look harmless; combined with internal domain names, table structures and error messages with full paths, it describes your information system.

The good practice is minimization: sending only what's strictly necessary for the reasoning. It's a skill in itself - building a minimal reproducible example with generic names - and it has an unexpected pedagogical virtue: isolating a problem to describe it abstractly forces you to understand it. The following prompt turns this confidentiality constraint into a diagnostic exercise.

Context: I'm working on proprietary code that I cannot share. I'll describe my problem abstractly, with generic names and without pasting the real code.

Description: [PSEUDO-CODE of the relevant structure, ERROR MESSAGE with paths and identifiers masked, VERSIONS of the libraries involved].

Your mission:
1. If my description is too vague to reason on, tell me exactly what information you're missing and why - I'll decide whether I can provide it in anonymized form.
2. Reason only on my description: never ask me to paste the full file.
3. Propose hypotheses for the cause, ranked by probability, and for each one a precise test I can run locally to confirm or rule it out.

Format: a table of hypothesis / probability / verification test. I'll come back with the test results.

Second file: generated-code licensing. The law isn't settled and varies by jurisdiction, but three facts are established. One: models are trained on code under a variety of licenses, including copyleft licenses like the GPL, and can reproduce snippets very close to their source - especially for famous algorithms or heavily duplicated snippets. Two: in several jurisdictions, a purely mechanical output with no human creative input is hard to protect by copyright, which undermines your claims over code accepted as-is. Three: some providers offer contractual indemnification in case of an infringement claim, and filters that block outputs too close to the corpus - check that these protections are in your plan, not just in the brochure. In practice: the longer, more specific and more unedited a generated snippet is, the higher the legal risk.

Third file: bias. A model reproduces the statistical regularities of its corpus, and three biases directly affect your technical decisions. The popularity bias: solutions over-represented in the data (dominant frameworks, patterns that were trendy two years ago) are recommended by default, even when your context calls for something else - the corpus is always one step behind the state of the art. The sycophancy bias: models are optimized to please the user and tend to validate your premise; ask "why is my microservices approach the right one?" and you'll get confirmations, not a counter-opinion. The inverted authority bias: the fluency of the tone produces an impression of certainty uncorrelated with reliability. The countermeasure is procedural: force the contradiction.

You just recommended [SOLUTION OR TECHNOLOGY] to me. Before I decide, switch stance: you are now an experienced engineer tasked with arguing AGAINST this recommendation.

1. Give the 3 best arguments against your own answer, each illustrated by a concrete scenario where it fails.
2. Cite at least one serious alternative you hadn't mentioned, and explain honestly why it was absent from your first answer.
3. Assess how much your initial recommendation owes to the popularity of this solution in your training data, rather than to my specific context, which I'll remind you of: [YOUR CONTEXT: team size, constraints, existing setup].

Format: two columns For / Against, then your revised recommendation in one sentence, with your confidence level and what could change it.

Knowledge check

Make sure you remember the key points of this lesson.

1. Before pasting your company code into an AI assistant, what should you check first?
   • That the snippet doesn't exceed the maximum context size
   • The provider's retention and training policy, as well as your contractual confidentiality obligations
   • That the code language is properly supported by the model
2. Why does model-generated code raise a licensing question?
   • Because all generated code is automatically placed under the GPL
   • Because the model can reproduce snippets very close to licensed code, whose obligations may then apply to your project
   • Because generated code always belongs to the model provider
3. What is a model's popularity bias?
   • The tendency to favor answers from the most active users
   • The tendency to recommend solutions over-represented in the training data rather than the one best suited to your context
   • The tendency to flatter the user by validating their premises