fscarmen

Video Tutorial#

Introduction and Project Features#

• This article introduces using GitLab as an image hosting service and GitHub as a backup.

• Previously, it was introduced how to use GitHub as an image hosting service and GitLab as a backup, 【GitHub Private Repository as Image Hosting, PicGO Upload, Cloudflare Workers Acceleration, GitLab Real-time Backup】, this is the reverse operation.

• Due to the extensive use of images in new blogs, a private image hosting service needs to be set up. It is necessary to avoid using paid commercial cloud OSS storage; find a long-term stable storage platform under the premise of free usage; also consider access speed; it is best to have off-site disaster recovery. To achieve these goals, the following comprehensive solution was developed.

Features and Advantages of the Solution:

• Stored in GitLab Private Repository: Images are stored in a GitLab private repository, with paths processed for desensitization to ensure security; the platform is even more stable than GitHub.
• Automatic Image Compression: Automatically compress images to improve loading speed and optimize user experience.
• Cloudflare CDN Acceleration: Utilize the global acceleration services provided by Cloudflare CDN to ensure fast image loading.
• Custom Domain Name: Supports the use of a custom domain name to enhance brand image and professionalism.
• Multiple Link Formats Output: Generates various forms of image links for easy use in blogs or other platforms.
• Real-time Backup to GitHub Private Repository (Optional): Real-time backup from GitLab to GitHub private repository provides data redundancy and cross-platform backup, enhancing data security and ensuring business continuity.

Requirements#

• GitLab Account, https://gitlab.com
• GitHub Account, optional, can be ignored if backup is not needed, https://github.com
• Cloudflare Account, https://www.cloudflare.com
• Tinypng Account, https://tinypng.com/developers
• Install Snipaste, https://zh.snipaste.com
• Install PicGo, https://molunerfinn.com/PicGo

Process and Tool Introduction#

Screenshot: Use Snipaste for screenshots, a simple yet powerful screenshot tool that supports screen capture, annotation, and more, suitable for Windows and Mac.

PicGo Upload Images to GitHub: Use PicGo and install the picgo-plugin-compress-next plugin to automatically compress images before uploading to GitHub.

Obtain Image Links Processed by Worker: Automatically generate API to hide GitLab private repository through Worker, add CDN and dual-stack, output custom domain URL.

Automatically Push Latest Images from GitHub to GitLab: Use GitLab CI/CD to automatically back up to GitHub whenever there is an update, avoiding the risk of losing all images.

Detailed Key Steps#

Prerequisites, Create GitLab Private Repository, Obtain GitLab API and Project ID#

• Log in to GitHub, visit https://gitlab.com/projects/new#blank_project to create a new project, and create a private repository for storing images. Currently, each project on GitLab is 4GB in size; if exceeded, just create a new one.

• Obtain the API for read and write permissions for the project.

• Obtain the Project ID.

Obtain Tinypng API Key#

• Register for a Tinypng.com account, visit https://tinypng.com/developers to register.

PicGo Settings#

• After installing PicGo, continue to install the picgo-plugin-compress-next plugin in the software (this step may require enabling scientific access).

• Install the gitlab-file plugin (this step may require enabling scientific access).

• Continue to set up the PicGO main program.

Create Worker in Cloudflare#

• Log in to Cloudflare, visit https://dash.cloudflare.com/, create a new worker.

• After deployment, edit the code, copy and modify the corresponding two locations as shown in the image.

// Define GitLab repository name, repository ID, and API token
const REPO_CONFIG = [
    { name: 'repoName1', id: 'repoID1', token: 'repoAPI1' },
    { name: 'repoName2', id: 'repoID2', token: 'repoAPI2' },
    { name: 'repoName3', id: 'repoID3', token: 'repoAPI3' },
  ];
  
  async function handleRequest(request) {
    const url = new URL(request.url);
    const pathParts = url.pathname.split('/').filter(Boolean);
  
    console.log('URL:', url.toString());
    console.log('Path parts:', pathParts);
  
    if (pathParts.length < 1) {
      return new Response('Invalid URL format', { status: 400 });
    }
  
    const gitlabRepo = pathParts[0];
    console.log('Requested repo:', gitlabRepo);
  
    const repoConfig = REPO_CONFIG.find(repo => repo.name === gitlabRepo);
  
    if (!repoConfig) {
      console.log('Repository not found in config. Available repos:', REPO_CONFIG.map(r => r.name));
      return new Response('Repository not found', { status: 404 });
    }
  
    const { id: gitlabRepoId, token: gitlabApiToken } = repoConfig;
  
    // Get the remaining path and encode it
    const remainingPath = pathParts.slice(1).join('/');
    const encodedPath = encodeURIComponent(remainingPath);
  
    console.log('Remaining path:', remainingPath);
    console.log('Encoded path:', encodedPath);
  
    // Build GitLab API URL
    const apiUrl = `https://gitlab.com/api/v4/projects/${gitlabRepoId}/repository/files/${encodedPath}/raw?ref=main`;
    console.log('GitLab API URL:', apiUrl);
  
    try {
      // Make a request to the GitLab API
      const response = await fetch(apiUrl, {
        method: 'GET',
        headers: {
          'PRIVATE-TOKEN': gitlabApiToken
        }
      });
  
      console.log('GitLab API response status:', response.status);
  
      if (!response.ok) {
        return new Response(`Error fetching from GitLab: ${response.statusText}`, { status: response.status });
      }
  
      // Return the response from GitLab
      const contentType = response.headers.get('Content-Type');
      const body = await response.arrayBuffer();
  
      console.log('Content-Type:', contentType);
  
      return new Response(body, {
        status: response.status,
        headers: {
          'Content-Type': contentType,
          'Cache-Control': 'public, max-age=3600' // Cache for one hour
        }
      });
    } catch (error) {
      console.error('Error:', error);
      return new Response(`An error occurred: ${error.message}`, { status: 500 });
    }
  }
  
  addEventListener('fetch', event => {
    event.respondWith(handleRequest(event.request));
  });

• Add a custom domain name, you can try the domain given by the worker https://&ltworkerName&gt.&ltcloudflareUser&gt.workers.dev first.

Official Usage Process#

Screenshot#

• Use Snipaste to take screenshots to the clipboard, you can use the shortcut key (default Ctrl + F1).

Upload Images#

• Upload images to GitHub via PicGo, you can use the shortcut key (default Ctrl + Shift + P) or drag and drop images into the PicGo interface.
• The upload success progress bar is blue; if it is red, please check if the settings are incorrect. After successful upload, the custom domain link will automatically be in the clipboard.
• Paste it in the appropriate scene, such as Blog or forum addresses.

Real-time Backup to GitHub (Optional)#

GitHub Settings#

• Log in to GitHub, create a backup repository, visit https://github.com/new.

• Visit https://github.com/settings/tokens?type=beta to obtain PAT.

GitLab Settings#

There are two methods, choose one: 1. Use the mirroring feature of the GitLab platform (recommended); 2. Use CI/CD workflow.

Method 1: GitLab Platform's Mirroring Feature#

Method 2: Through CI/CD Pipeline#

• Create three environment variables in CI/CD.

• Create a CI/CD Pipeline to back up to GitHub, the code is as follows:

image: alpine:latest

variables:
  GIT_STRATEGY: clone

before_script:
  - apk add --no-cache curl git

sync_to_github:
  script:
    - |
      # Get the latest commit hash from local and GitHub remote repositories and display
      LOCAL_COMMIT=$(git rev-parse HEAD)
      REMOTE_COMMIT=$(curl -s -H "Authorization: token ${GITHUB_PAT}" "https://api.github.com/repos/${GITHUB_USERNAME}/${GITHUB_REPO}/commits/main" | awk -F '"' '/"sha":/{print $4; exit}')
      echo "Local commit: $LOCAL_COMMIT"
      echo "Remote commit: $REMOTE_COMMIT"

      # Compare local and remote commits, if updated, push to GitHub
      if [ "$LOCAL_COMMIT" != "$REMOTE_COMMIT" ]; then
        echo "Pushing updates to GitHub..."
        git config --global user.email "${GITHUB_USERNAME}@gitlab.com"
        git config --global user.name "${GITHUB_USERNAME}"
        git remote add github https://${GITHUB_PAT}@github.com/${GITHUB_USERNAME}/${GITHUB_REPO}.git
        git push github HEAD:main --force
      else
        echo "No updates needed."
      fi
  rules:
    - if: $CI_PIPELINE_SOURCE == "push" || $CI_PIPELINE_SOURCE == "web"

Image Example#

https://pic.forvps.gq/pic2/images/202410081627933.png