How to Convert a Website URL Page to Text using Go

When we want to strip plain, unformatted text (and nothing else) from a web page, all we’re really trying to do is remove text from the HTML code it’s enclosed within. If we don’t have the required HTML string readily available, our best bet to perform this task is to use the web page URL instead.

Our URL to TXT API condenses the HTML-to-text conversion process by retrieving the HTML string associated with a given URL and subsequently removing all plain, unformatted text from it automatically. This API’s request format should look like the below JSON example (or the XML equivalent):

{
  "Url": "string"
}

The text from your URL page will be returned as a string, which can be copied or written to any file format you see fit.

This API is free & easy to use; all you need to do is follow the complementary instructions provided below to structure your API call with ready-to-run Go code examples. Before you do so, however, you’ll first need to register a free account on our website to get your free-tier API key (this will yield a limit of 800 API calls per month).

In order to call this API, simply include the below examples in your file and provide your API key string & URL input request in the appropriate fields:

package main

import (
     "fmt"
     "strings"
     "net/http"
     "io/ioutil"
)

func main() {

     url := "https://api.cloudmersive.com/convert/web/url/to/txt"
     method := "POST"

     payload := strings.NewReader("Url=%3Cstring%3E")

     client := &http.Client {
     }
     req, err := http.NewRequest(method, url, payload)

     if err != nil {
          fmt.Println(err)
          return
     }
     req.Header.Add("Content-Type", "application/x-www-form-urlencoded")
     req.Header.Add("Apikey", "YOUR-API-KEY-HERE")

     res, err := client.Do(req)
     if err != nil {
          fmt.Println(err)
          return
     }
     defer res.Body.Close()

     body, err := ioutil.ReadAll(res.Body)
     if err != nil {
          fmt.Println(err)
          return
     }
     fmt.Println(string(body))
}

After that, you’re all done. It’s just that easy!