GitHub Copilot Certification Exam Prep

There is an exciting 4-Week Course on GitHub Community to help developers master GitHub Copilot & prepare for the GitHub Copilot Certification Exam.

There is a very useful curation of study material spanning the 7 Domains that will be covered in the exam & a knowledge check with practice questions for each of the 4 weeks.

I plan to compile the reasoning for the answers to the practice questions here so that the notes can be quick reference for the exam & later -

Week 1

1. GitHub Copilot is trained on publicly available open-source repositories and large proprietary datasets curated by GitHub. It does not use private repositories of individual users. Documentation and Stack Overflow discussions are not primary data sources for Copilot.
2. GitHub Copilot enhance a developer's productivity during pair programming by providing inline code suggestions based on context
3. Key ethical considerations when using GitHub Copilot for software development:
1. Ensuring the code adheres to licensing requirements. Licensing issues are critical when using AI-generated code. Copilot may generate code based on public repositories or other code sources, so it’s essential to ensure that the generated code complies with licensing and usage terms.
2. Regularly reviewing AI-generated code for potential bias
4. To mitigate security risks when using Copilot developers should conduct thorough code reviews before deployment to ensure that no security vulnerabilities or errors are introduced. Relying solely on AI suggestions without validation can pose risks to the integrity and security of the project.
5. Features exclusive to GitHub Copilot Enterprise compared to Business
1. Integration with Microsoft Entra ID (Azure AD) to manage authentication and access for enterprise users
2. Admin-level policy management allows enterprise admins to have finer control over the usage of Copilot in their organization
6. Developers can maximize the accuracy of GitHub Copilot's code suggestions by writing clear and descriptive function names and comments. It ensures that Copilot can generate contextually appropriate and semantically correct code, boosting productivity and code quality.
7. The following actions reflect responsible AI usage with GitHub Copilot:
1. Using code suggestions as a reference and refining manually
2. Ensuring the AI-generated code meets compliance standards
8. The GitHub Copilot for Business subscription plan allows GitHub Copilot usage in a business environment. It offers features like centralized billing and team management, which are not available in the individual plans.
9. Steps to enable Copilot in VS Code:
1. Install the GitHub Copilot extension
2. Link a GitHub account with an active Copilot subscription
10. GitHub Copilot handles different programming paradigms by adapting suggestions based on project context and code style
11. GitHub Copilot contributes to responsible AI usage by:
1. Providing proper attribution for AI-generated code to maintain transparency and respect intellectual property rights
2. Encouraging continuous user feedback to refine AI behavior, enhance accuracy, and mitigate potential biases in suggestions
12. Recommended practice for using GitHub Copilot effectively in a team setting:
1. Establishing clear guidelines for Copilot usage
2. Regular knowledge-sharing sessions on Copilot best practices

Week 2

1 A) GitHub Copilot for Enterprise allows organizations to configure custom LLM training with proprietary data. With GitHub Copilot Enterprise, you can fine-tune a private, custom model, which is built on a company’s specific knowledge base and private code. Organizations that utilize GitHub Copilot Enterprise’s custom models enable more accurate and contextually relevant suggestions and responses. Custom models for GitHub Copilot Enterprise is in public preview and is subject to change.

Copilot Business and Enterprise both support SOC 2 Type II framework

Both Copilot Business and Enterprise ensures zero data retention for AI-generated completions. The GitHub Copilot extension in the code editor does not retain your prompts for any purpose after it has provided Suggestions, unless you are a Copilot Pro or Copilot Free subscriber and have allowed GitHub to retain your prompts and suggestions. 

2 A,D) GitHub Copilot does NOT provide a suggestion when the user is writing a private, proprietary API call with no prior examples in public repositories and also when the user is using Copilot in VS Code, but their organization has Copilot completions disabled at the repository level. This is an administrative restriction that overrides user-level settings

While hitting a rate limit might temporarily pause your ability to use Copilot Chat. This would be a temporary condition, not a scenario where Copilot would never provide suggestions. Once the rate limit period expires, suggestions would resume. This is not a definitive scenario where Copilot would NOT provide a suggestion.

3 A) Agent Mode can automatically recognize and correct errors in its own code suggestions.

4 D) Copilot filters sensitive data using a heuristic-based approach before processing. The pre-processed prompt is then passed through the Copilot Chat language model, which is a neural network that has been trained on a large body of text data. 

As suggestions are generated and before they are returned to the user, Copilot applies an AI-based vulnerability prevention system that blocks insecure coding patterns in real-time to make Copilot suggestions more secure. Our model targets the most common vulnerable coding patterns, including hardcoded credentials, SQL injections, and path injections.

Input prompts and output completions are run through content filters.

Copilot processes prompts flagged for PII. This happens on the Proxy server hosted on GitHub-owned Azure tenants.

5 A) The suggestion most frequently accepted by developers in similar contexts is ranked higher. Simply matching common patterns isn't the primary driver for ranking suggestions. Copilot balances common patterns with the specific context of the code being written. 

To generate a code suggestion, the Copilot extension begins by examining the code in your editor_—_focusing on the lines just before and after your cursor, but also information including other files open in your editor and the URLs of repositories or file paths to identify relevant context. That information is sent to Copilot’s model, to make a probabilistic determination of what is likely to come next and generate suggestions.  

To generate a suggestion for chat in the code editor, the Copilot extension creates a contextual prompt by combining your prompt with additional context including the code file open in your active document, your code selection, and general workspace information, such as frameworks, languages, and dependencies. That information is sent to Copilot’s model, to make a probabilistic determination of what is likely to come next and generate suggestions.

6 A) The Copilot Proxy plays a crucial role in the GitHub Copilot data pipeline by routing and processing user requests before sending them to the LLM.

7 A,D) A user can experience delayed Copilot completions if the IDE's context window exceeds Copilot's processing limit and Copilot’s rate limits have been exceeded, causing temporary delays.

8 A) GitHub Copilot is most likely to produce incorrect or “hallucinated” code when low-confidence completions are not filtered out. Low-confidence completions occur when Copilot lacks sufficient training data, leading to incorrect or unpredictable results.

9 B) When using Copilot-generated code be aware that Copilot does not automatically sanitize user input, increasing the risk of injection attacks.

10 B, D) GitHub Copilot Chat is supported in JetBrains IDEs, Visual Studio, Visual Studio Code & Xcode. These platforms support AI-powered chat, allowing developers to ask coding-related questions, request explanations, and generate code directly in their development environment. 

GitHub Codespaces and the GitHub Web UI do not support Copilot Chat at this time. You can use GitHub Copilot in GitHub Codespaces by adding a VS Code extension but it does not currently support Copilot Chat functionality.

11 B) A key limitation of Copilot’s CLI integration compared to IDE-based Copilot features is that it lacks access to context from open files in the IDE. This limits its ability to provide deeply relevant suggestions. It does not exclusively work with Git commands, automatically commit, or restrict itself to single-line commands.

12  A,C ) The key limitations of GitHub Copilot’s LLM-based code generation are that it struggles with complex multi-step reasoning, often requiring developer intervention for logical correctness and it can produce incomplete or syntactically incorrect suggestions, especially in low-resource programming languages. GitHub Copilot’s LLM-based code generation is powerful but has limitations in complex multi-step problem-solving, often requiring developer oversight to ensure logical correctness. Additionally, in low-resource programming languages, Copilot’s training data may be insufficient, leading to incomplete or incorrect syntax. Its suggestions are not deterministic, and it does not guarantee prevention of licensing conflicts with GPL-licensed code.

13 C) The recommended best practice for an organization implementing Copilot for Business is to configure organization-wide policies to enforce responsible AI usage. 

14 A,C) Copilot for Enterprise be customized for an organization’s internal workflows by training Copilot on internal, proprietary codebases for better context and configuring Copilot Knowledge Bases for internal documentation lookups. 

GitHub Copilot for Enterprise allows organizations to enhance AI-generated suggestions by training Copilot on proprietary codebases, ensuring better alignment with internal best practices. 

Additionally, Copilot Knowledge Bases help integrate internal documentation, allowing developers to query internal workflows, coding standards, and best practices efficiently. Restricting Copilot to pre-approved open-source licenses or limiting it to past completions are not supported methods of customization.

Additional Notes -

• In 2021, OpenAI released the multilingual Codex model, which was built in partnership with GitHub. 
• GitHub Copilot launched as a technical preview in June 2021 and became generally available in June 2022 as the world’s first at-scale generative AI coding tool.
• Codex contains upwards of 170 billion parameters.
• GitHub Copilot gathers context from:
• Code after cursor
• File name and
• Other open tabs in the editor
• GitHub Copilot API, the GitHub Copilot LLMs are hosted in GitHub-owned Azure tenants. These LLMs consist of AI models created by OpenAI that have been trained on natural language text and source code from publicly available sources, including code in public repositories on GitHub.
• OpenAI's GPT-4 model adds support in GitHub Copilot for AI-powered tags in pull-request descriptions through a GitHub app that organization admins and individual repository owners can install. GitHub Copilot automatically fills out these tags based on the changed code. Developers can then review or modify the suggested descriptions.
• Both GitHub Copilot Business & GitHub Copilot Enterprise have IP indemnity and enterprise-grade security, safety, and privacy.
• GitHub Copilot Enterprise can index an organization's codebase for a deeper understanding and for suggestions that are more tailored. It offers access to GitHub Copilot customization to fine-tune private models for code completion.
• GitHub Copilot X aims to bring AI beyond the IDE to more components of the overall platform, such as docs and pull requests.
• GitHub Copilot is available as an extension for VS Code, Visual Studio, Vim/Neovim, JetBrains suite of IDEs, Azure Data Studio, Xcode.
• Languages with less representation in public repositories may be more challenging for Copilot Chat to provide assistance with. 
• In Copilot Chat, if a particular request is no longer helpful context, delete that request from the conversation. Alternatively, if none of the context of a particular conversation is helpful, start a new conversation.
• GitHub Copilot transmits data to GitHub’s Azure tenant to generate suggestions, including both contextual data about the code and file being edited (“prompts”) and data about the user’s actions (“user engagement data”). The transmitted data is encrypted both in transit and at rest; Copilot-related data is encrypted in transit using transport layer security (TLS), and for any data we retain at rest using Microsoft Azure’s data encryption (FIPS Publication 140-2 standards).
• Copyright law permits the use of copyrighted works to train AI models. GitHub Copilot’s AI model was trained with the use of code from GitHub’s public repositories - which are publicly accessible and within the scope of permissible copyright use.
• GitHub Copilot users should align their use of Copilot with their respective risk tolerances. It is your responsibility to assess what is appropriate for the situation and implement appropriate safeguards. 
• GitHub does not claim ownership of a suggestion.
• As suggestions are generated and before they are returned to the user, Copilot applies an AI-based vulnerability prevention system that blocks insecure coding patterns in real-time to make Copilot suggestions more secure. Our model targets the most common vulnerable coding patterns, including hardcoded credentials, SQL injections, and path injections. The system leverages LLMs to approximate the behavior of static analysis tools and can even detect vulnerable patterns in incomplete fragments of code. This means insecure coding patterns can be quickly blocked and replaced by alternative suggestions. The best way to build secure software is through a secure software development lifecycle (SDLC). GitHub offers solutions to assist with other aspects of security throughout the SDLC, including code scanning (SAST), secret scanning, and dependency management (SCA). GitHub recommends enabling features like branch protection to ensure that code is merged into your codebase only after it has passed your required tests and peer review.
• As with any code that your developers did not originate, the decision about when, how much, and in what context to use any code is one your organization needs to make based on its policies, and in consultation with industry and legal service providers. All organizations should maintain appropriate policies and procedures to ensure that these licensing concerns are properly addressed.