git.qmd

---
title: "Git, GitLab, and GitHub"
---

For code below, Open `Git` (or Powershell, Terminal, Command Prompt, etc.) in directory of repository and use the relevant code.

# When to Use GitHub vs GitLab {#sec-GitHubVsGitLab}

We use repositories for version control of code and data files (see the next section for more information on why).
For data security, we use the university's Enteprise instance of `GitLab` (<https://research-git.uiowa.edu>) for repositories that include data files.
For repositories that we want to generate a public website from (and that do NOT include study data), we use `GitHub` (<https://github.com>).

For repositories to save in `GitLab`, save the repository in the relevant subproject folder of the PetersenLab Group: <https://research-git.uiowa.edu/PetersenLab>.
For repositories to save in `GitHub`, save the repository in the DevPsyLab Organization group: <https://github.com/DevPsyLab>.

However, regardless of whether you will store the repository in `GitHub` or `GitLab`, you will still use `git` for version control via `GitHub Desktop`.
For information on how to set up and use `git` and `GitHub Desktop`, see Section \@ref(howToUseGit).

# Why It is Important to Use `Git`/Version Control {#sec-whyGit}

There are many reasons why it is important to use git/version control:

- It (`GitHub`/`GitLab`) provides backups of all files in the cloud
- It (`GitHub`/`GitLab`) allows you to access the files in the cloud from any computer with access to the internet, even if you are away from your local computer
- Version control provides history of all (committed) changes made to all files
- If you make a mistake in your code, you can easily revert to a prior version because you took snapshots (commits) of your files!
    - You don't need to keep all the "commented out" lines of code
        - This allows you to write cleaner code, because you can remove commented out lines of code, knowing that you can easily recover them in your commit history
    - You don't have to keep files or folders named "project_old" or whatever
        - This allows you to keep cleaner file/folder directories for your projects
- `Git` keeps all files for the project self-contained, which helps increase reproducibility
- It (`GitHub`/`GitLab`) allows easier collaboration with others on the project
- It (`GitHub`/`GitLab`) allows easier sharing with outside researchers

# How to Use {#sec-howToUseGit}

## To begin {#sec-toBegin}

1. Install `git` (<https://git-scm.com/downloads>)
1. Install `GitHub Desktop` app (<https://desktop.github.com>)
1. Create a Personal Access Token (<https://docs.gitlab.com/ee/user/profile/personal_access_tokens.html>, archived at <https://perma.cc/6ZH8-JNZC>; or <https://research-git.uiowa.edu/-/profile/personal_access_tokens>, archived at <https://perma.cc/7JPU-WDBS>) on the UI `GitLab` instance (<https://research-git.uiowa.edu>) to access UI `GitLab` repositories from `GitHub Desktop`.
    -  For application name, put "GitHub"
    -  For expiration date, leave it blank (or set it for as late an expiration date as possible)
    -  For scopes, select everything
1. When cloning a repository from UI's `GitLab` instance using `GitHub Desktop`, `GitHub Desktop` will ask for your username and password.
Enter your HawkID as your username (or your GitHub.com username if different) and your UI `GitLab` Personal Access Token as your password.

## Best practices working with version control {#sec-bestPractices}

- Create a new repository (repo) whenever you start a new project
- Follow the Petersen Lab template for how to structure your repo (folder structure, `.gitignore` file, etc.):
    - <https://research-git.uiowa.edu/PetersenLab/Template>
- To collaborate with others:
    - Navigate to the repo on the UI `GitLab` website
    - When in the repo, click "Members"
    - Add the Collaborator
- Each time you want to work on the files in the repo, follow this cycle:
    1. Using the `GitHub Desktop` app, sync the repo files from the cloud to the repo on your local computer (i.e., fetch any repo updates to your local machine from the cloud)
        - Make sure to do this before many code changes so you are working with the latest version of files
    1. Do your work on the repo: make any code/file/folder additions, changes, or deletions
    1. Using the `GitHub Desktop` app, commit the changes
        - Commit changes to the cloud early and often; when deciding what to commit and when, try to group "similar changes" into the same commit ("like goes with like")
        - Use a separate "commit" for each separable "functional unit" changed
    1. Using the `GitHub Desktop` app, sync the repo files from your local computer to the repo on the cloud (i.e., push all of your committed changes to the cloud)
        - Never leave file changes uncommitted or unsynced when you stop working on the repo for the day!

## Instructions for using `GitHub Desktop` app with `GitLab`

- <https://itnext.io/how-to-use-github-desktop-with-gitlab-cd4d2de3d104> (archived at <https://perma.cc/S59R-3YT7>)
- <https://community.reclaimhosting.com/t/using-github-desktop-with-gitlab/876> (archived at <https://perma.cc/ZPT9-KEQL>)
- <https://stackoverflow.com/questions/22639815/does-github-for-windows-work-with-gitlab> (archived at <https://perma.cc/99RR-6KLE>)
- <https://github.com/desktop/desktop/issues/852#issuecomment-402546848> (archived at <https://perma.cc/3GCU-GTPU>)
- <https://github.com/desktop/desktop/issues/3816#issuecomment-421060974> (archived at <https://perma.cc/N8RS-BFWR>)

## How to create a repo on local computer

1. Create repository on UI `GitLab` website (<https://research-git.uiowa.edu>)
1. Open `GitHub Desktop` app
1. Clone repository
1. Add `.gitignore` file (from Template project: <https://research-git.uiowa.edu/petersenlab/Template/blob/master/.gitignore>) to the root of the cloned project folder
1. Follow the Petersen Lab template for how to structure your repo (folder structure, `.gitignore` file, etc.): <https://research-git.uiowa.edu/PetersenLab/Template>
    -  Note that a folder will not be synced if there are no files in them (i.e., if the folder is empty)
1. Sync file changes using `GitHub Desktop` ("Fetch origin", "Push origin")

## How to add a pre-existing repo from the lab drive (`LSS`/`RDSS`/network share) to your computer {#sec-preExistingRepoLabDrive}

For example, the `SRS-DataProcessing` repo lives on the lab drive.

1. Make sure Dr. Petersen has given you collaborator access to the repo
1. Open `GitHub Desktop` app
1. Navigate to the folder location of the repo on the lab drive
1. Drag the `.git` folder within the repo to the `GitHub Desktop` app

## How to clone a repo from `GitLab` to local

1. Navigate to relevant repo on `GitLab` (<https://research-git.uiowa.edu>)
1. Click `Clone` in right-hand corner, select `Clone with HTTPS`
1. Open `GitHub Desktop` App and click `File`, `Clone Repository`
1. Click the `URL` tab
1. Paste in the URL
1. Navigate to where you want to save it
    -  The recommended location for your repos is to create a folder titled `GitHub` in your `Documents` folder, and to put repos in the `GitHub` folder (by default: PC: `C:/Users/[USERNAME]/Documents/GitHub/`; Mac: `/Users/[user]/Documents/GitHub/`) because various lab scripts try to read the lab functions from this location; it is *NOT* recommended to put `git` repos in a OneDrive folder because [`git` files tend not to play nice with syncing services](<https://stackoverflow.com/questions/19305033/why-is-putting-git-repositories-inside-of-a-dropbox-folder-not-recommended>; archived at <https://perma.cc/UTX8-KVL9>) (e.g., OneDrive, Dropbox)
1. Click `Clone`
1. Enter your `GitLab` username as your username and your Personal Access Token as your password

## How to add, modify, or delete files in a repo

1. Open relevant repo in `GitHub Desktop` app
1. Pull any repo updates from the server to the local files ("Fetch origin", "Pull origin")
1. Make necessary additions, modifications, and deletions to the files
1. Create commits for all changes in `GitHub Desktop` app (one commit per substantive change): Enter "Summary" and "Commit to master"
1. After making all changes and commits, push local file changes to the server using `GitHub Desktop` ("Fetch origin", "Push origin")

## How to collaborate with others

1. Navigate to the repo on the UI `GitLab` website
1. When in the repo, click "Settings"
1. Click "Members"
1. Add the Collaborator

## How to create a pull request {#sec-pull-request}

To make/propose changes to a repo that you do not have write permissions for, you need to create a pull request.

1. To create a pull request to a repo that you do not have write permissions for, first create a fork of the repo.
To fork the repo, see here: <https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/fork-a-repo#forking-a-repository> (archived at <https://perma.cc/33P6-Y4BR>)
1. After forking the repo, sync the repo with the latest version from the cloud so you know you are editing the latest version of the files.
When asked by GitHub Desktop how you plan to use the fork, select "To contribute to the parent project", as in the image below.
![fork](images/fork.png)
1. Make any changes to the files in the repo that you'd like to incorporate into the repo.
1. Open a pull request with your changes: <https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/creating-a-pull-request-from-a-fork> (archived at <https://perma.cc/KAP3-L5L7>)
1. The owner of the repo will review your changes and decide whether to accept them or whether additional changes are needed.

You can also create a pull request if you have write permissions to the repo.
This can be useful when collaborating on code with multiple people.
To do this, first, create a new branch.
Then, make your changes in the new branch.
After making your changes, create a pull request to merge the changes in the new branch into the main branch.

Note: If your pull request is showing up in your local repo but not in the original repo (for the repo owner), you might need to delete your forked repo and re-fork the original repo.

## How to clone a repo into local directory with a different folder name (directory must be empty)

1. `Git` Bash into directory
```
git clone https://research-git.uiowa.edu/petersenlab/srs/SRS-DataProcessing.git .
git remote set-url --add origin https://research-git.uiowa.edu/petersenlab/srs/SRS-DataProcessing.git
git remote -v
```

## How to transfer a repo to a new location/group/subgroup

1. Create location/group/subgroup (e.g., `PetersenLab/School Readiness Study`)
1. When in the repo, click "Settings"
1. Go to "Advanced", and click "Expand"
1. Go to "Transfer project", and select the location/group/subgroup you want to transfer the repo to under "Select a new namespace"
1. In the local repo, edit the repo URL in the `.git/config` file
1. When asked for your password, enter your username (HawkID) and `GitLab` Personal Access Token

## How to use large file storage (LFS) {#sec-gitLfs}

In `GitLab`, we use large file storage (LFS) to store large files.
In `GitHub`, we do not use LSS to store large files, because the University does not have an Enterprise agreement with `GitHub` (thus, we would have to pay to use it).
As a consequence, individual files hosted in a `GitHub` repo must be less than 50 MB in size.
By contrast, individual files hosted in a `GitLab` repo may be larger than 50 MB.

To set up LFS in for a `GitLab` repo, follow these steps:

1. Make sure the large files are not in the repo yet.
1. In `GitHub Desktop`, open the repo you want to use LFS for
1. In `GitHub Desktop`, select the `Repository` tab, then select `Command Prompt` or similar
1. In the command prompt, type (based on instructions from: <https://docs.gitlab.com/ee/topics/git/lfs/>; archived at <https://perma.cc/6WMC-GTKN>):
```
git lfs install          # initialize the Git LFS project
git lfs track "*.Rdata"  # select the file extensions that you want to treat as large files
```
5. This should have created a `.gitattributes` file in the repo. In `GitHub Desktop`, commit and push the `.gitattributes` file to the cloud version of the repo.
6. Copy the large files into the repo.
7. In `GitHub Desktop`, commit and push the large file to the cloud version of the repo.
8. (Or, if not using `GitHub Desktop`, can commit in the command prompt (<https://docs.gitlab.com/ee/topics/git/lfs/>)):
```
git add .                                            # add the large file to the project
git commit -am "insert name of commit message here"  # commit the file meta data
git push origin master                               # sync the git repo and large file to the GitLab server
```
9. When asked for credentials, use your HawkID as your username and your personal access token as your password

## How to create a new repository on a shared network drive (e.g., Lab Drive)

1. Create a repo on `GitHub` or `GitLab` online
1. Open `Git` Bash on the desktop
1. Using `Git` Bash, set the current directory to the path where the new repo will be.
For example, use the following command to clone to the School Readiness Study

```
cd "R:\Lab\Studies\School Readiness Study"
```

1. Next, go to `GitLab` or `GitHub` online and obtain the HTTPS URL to the new repository that was created.
Clone the repository with the HTTPS link using `Git` Bash with the following command

```
git clone (HTTPS Link)
```

## How to revert changes to a previous commit {#sec-revertCommit}

1. Open the `Github Desktop` and navigate to the repository you would like to revert changes for
1. Next, click the `history` tab
1. Right click the commit you would like to revert back to, and select revert changes
1. Push the changes to the repo to complete the revision

![revert changes](images/revertChanges.png)

## How to perform a partial commit {#sec-partialCommit}

1. In `GitHub Desktop`, click the desired lines in the gutter
2. Create the commit, and leave the other changes for you to continue working on.

<https://github.blog/2015-01-14-partial-commits-in-github-for-windows> (archived at <https://perma.cc/5U3V-YWRF>)

## How to copy a repo to a new repo {#sec-copyRepo}

<https://github.com/new/import>

## How to make a repository a template repository {#sec-templateRepo}

<https://docs.github.com/en/repositories/creating-and-managing-repositories/creating-a-template-repository> (archived at <https://perma.cc/PYW5-KAP5>)

## How to create a respository from a template {#sec-repoFromTemplate}

<https://docs.github.com/en/repositories/creating-and-managing-repositories/creating-a-repository-from-a-template> (archived at <https://perma.cc/9E2C-MUCK>)

# Troubleshooting {#sec-troubleshootingGit}

A troubleshooting guide for Git is located here:

<https://ohshitgit.com> (archived at: <https://perma.cc/7E6X-JQW2>)

## Error: Authentication Failed {#sec-authenticationFailed}

![authentication failed](images/githubDesktopAuthenticationError.png)

This error can occur for a variety of reasons.
One possibility is that your GitLab Personal Access Token (PAT) has expired.
To fix this:

1. Create a Personal Access Token (<https://docs.gitlab.com/ee/user/profile/personal_access_tokens.html>, archived at <https://perma.cc/6ZH8-JNZC>; or <https://research-git.uiowa.edu/-/profile/personal_access_tokens>, archived at <https://perma.cc/7JPU-WDBS>) on the UI `GitLab` instance (<https://research-git.uiowa.edu>) to access UI `GitLab` repositories from `GitHub Desktop`.
    -  For application name, put "GitHub"
    -  For expiration date, leave it blank (or set it for as late an expiration date as possible)
    -  For scopes, select everything
1. When cloning a repository from UI's `GitLab` instance using `GitHub Desktop`, `GitHub Desktop` will ask for your username and password.
Enter your HawkID as your username and your UI `GitLab` Personal Access Token as your password.

If you do not receive a prompt to enter your username and password, try removing the saved GitHubcredentials from Windows Credential Manager:
<https://github.com/desktop/desktop/issues/8860#issuecomment-2211812646>

## GitHub trying to connect to `git config` in another user's User folder

To resolve, ensure that the `HOME` environment variable is set to your user folder (assuming that it is not a shared computer)

To do so:
   - Type "Environment Variables" into the search bar of your Windows task bar
   - Click "Open" under the result of "Edit the system environment variables"
   - Enter an admin user/password when prompted
   - Click "Environment Variables" at the bottom right corner of the popup window that should open after entering admin credentials
   - Find the `HOME` environment variable and click it to select (highlighted blue)
   - Click "Edit" and update the path to match your Users folder


## `GitHub` shows all files as being changed even though the files haven't changed

Windows and Mac use different line endings (<https://github.com/Microsoft/WSL/issues/184>; archived at <https://perma.cc/F8UX-YJP3>):

```
git config --global core.autocrlf true
```
  
## Error: `You are not allowed to push code to protected branches on this project`

1. Make sure the owner of the repo unprotects the branch (<https://stackoverflow.com/questions/32246503/fix-gitlab-error-you-are-not-allowed-to-push-code-to-protected-branches-on-thi>; archived at <https://perma.cc/98AF-N6BY>)
1. Open the repo, click "Settings", "Repository", "Protected Branches"
1. Change "Allowed to merge" and "Allowed to push" to "Developers and Maintainers", and click "Unprotect"

## Error creating commit

```
git add -A
git status
git commit -m "Message"` (where "Message" is the summary message of the commit)
git push
git status
```

## Undo commit (but retain file changes)

```
git reset --soft HEAD^
```

## Error: `Sync failed`

```
git status
git push
```

## Error: `Sync failed -- Syncing would overwrite your uncommitted changes`

<https://stackoverflow.com/questions/23084822/github-some-uncommited-changes-would-be-over-written-by-syncing> (archived at <https://perma.cc/AP5S-G38B>):

1. `git stash -u`
1. (perform manual sync in `GitHub Desktop`)
1. `git stash pop`
  
## Error: `Pipeline has failed for master`

<https://docs.gitlab.com/ee/topics/autodevops/#at-the-project-level> (archived at <https://perma.cc/3MAQ-ZBFF>)

1. Open the repo in `GitLab`
1. Go to your project's "Settings" > "CI/CD" > "Auto DevOps"
1. Disable (uncheck) the "Default to Auto DevOps pipeline"
1. Click "Save changes"

# OLD

## Creating Repo on Local Directory (if Directory is empty)

1. Create Repo on GitHub
1. Don't add README yet
1. Open `Git` Shell, navigate to directory, and type:
```
git init
git remote add origin https://github.com/DevPsyLab/petersenlab.git
git remote -v
```
4. Drag and drop the folder with the repository into the GUI app
5. Add .gitignore file with .Rhistory

## Creating Repo on Local Directory (if Directory is *not* empty)

1. Create Repo on GitHub
1. Don't add README yet
1. Open `Git` Shell, navigate to directory, and type:
```
git init
git add .
git commit -m 'First commit'
git remote add origin https://research-git.uiowa.edu/itpetersen/PetersenLab.git
git remote -v
git push -u origin master
```
4. Drag and drop the folder with the repository into the GUI app
5. Add .gitignore file with .Rhistory

## Creating `R` Scripts Repo on Lab Server

1. Create `R` Scripts Repo
1. Open `R` Scripts folder and delete `.git` folder
1. Open `Git` Shell and type:
```
git init
git add .
git commit -m 'First commit'
git remote add origin https://research-git.uiowa.edu/PetersenLab/R-Scripts.git
git remote -v
git push -u origin master
```
4. Move `R` Scripts folder to another location
5. Clone repo into folder:
    - open `Git` Shell
    - navigate to `Z:\TDS II\Data\R Scripts\`
    ```
    git clone https://research-git.uiowa.edu/PetersenLab/R-Scripts.git .
    ```
    The dot on the end of the `git clone` command means "the current directory"
6. Drag and drop the folder with the repository into the GUI app
7. Add `.gitignore` file with `.Rhistory`