chromedp: Get started to surfing the web with Go

chromedp helps you to flip through! the web fast, easy, and programmatically in Golang. If you have ever worked with Selenium or PhantomJS or other similar tools, this concept is familiar to you.

In this article we are going to get start working with chromedp and do some simple tasks with it. Let's get started.

First thing first you have to add chromedp to your project's dependency. It can done just like other golang packages that you used before:

go get -u github.com/chromedp/chromedp

for starting work with chromedp, you do not need to do a lot! the only thing that you need to do is creating a context and start working with it;

ctx, cancel := chromedp.NewContext(context.Background())
defer cancel()
chromedp.Run(ctx, chromedp.Navigate("https://www.google.com"))

That's it. Trust me, chromedp just open google even if you don't see anything yet! the reason that you didn't see the browser is by default it runs in headless mode and as I expected we can change the behavior. Actually, there are a list of default behaviors that passed to the library by default

// DefaultExecAllocatorOptions are the ExecAllocator options used by NewContext
// if the given parent context doesn't have an allocator set up. Do not modify
// this global; instead, use NewExecAllocator. See ExampleExecAllocator.
var DefaultExecAllocatorOptions = [...]ExecAllocatorOption{
    NoFirstRun,
    NoDefaultBrowserCheck,
    Headless,

    // After Puppeteer's default behavior.
    Flag("disable-background-networking", true),
    Flag("enable-features", "NetworkService,NetworkServiceInProcess"),
    Flag("disable-background-timer-throttling", true),
    Flag("disable-backgrounding-occluded-windows", true),
    Flag("disable-breakpad", true),
    Flag("disable-client-side-phishing-detection", true),
    Flag("disable-default-apps", true),
    Flag("disable-dev-shm-usage", true),
    Flag("disable-extensions", true),
    Flag("disable-features", "site-per-process,TranslateUI,BlinkGenPropertyTrees"),
    Flag("disable-hang-monitor", true),
    Flag("disable-ipc-flooding-protection", true),
    Flag("disable-popup-blocking", true),
    Flag("disable-prompt-on-repost", true),
    Flag("disable-renderer-backgrounding", true),
    Flag("disable-sync", true),
    Flag("force-color-profile", "srgb"),
    Flag("metrics-recording-only", true),
    Flag("safebrowsing-disable-auto-update", true),
    Flag("enable-automation", true),
    Flag("password-store", "basic"),
    Flag("use-mock-keychain", true),
}

As you can see, the third item in the array is Headless so we can modify it. Let's create our configs for creating a new context.

opts := append(
  // select all the elements after the third element
    chromedp.DefaultExecAllocatorOptions[3:],
    chromedp.NoFirstRun,
    chromedp.NoDefaultBrowserCheck,
)

the latest step is to create an ExecAllocator and pass it as our parent context.

parentCtx, _ := chromedp.NewExecAllocator(context.Background(), opts...)
ctx, _ := chromedp.NewContext(parentCtx)

Now if you run the project you will see the browser open and navigates to the google website. here's the complete code:

package main

import (
    "context"
    "fmt"

    "github.com/chromedp/chromedp"
)

func main() {
    opts := append(
        // select all the elements after the third element
        chromedp.DefaultExecAllocatorOptions[3:],
        chromedp.NoFirstRun,
        chromedp.NoDefaultBrowserCheck,
    )
    // create chromedp's context
    parentCtx, cancel := chromedp.NewExecAllocator(context.Background(), opts...)
    defer cancel()

    ctx, cancel := chromedp.NewContext(parentCtx)
    defer cancel()

    if err := chromedp.Run(ctx, chromedp.Navigate("https://www.google.com")); err != nil {
        panic(err)
    }

    fmt.Println("I've just saw Google!!!")
}