How to run a run a BigQuery job programmatically

Question

I have a big query job that I need to run programmatically. How do I do it?

Answer 1

To run a job programmatically:

1. Start the job by calling the jobs.insert method. When you call the jobs.insert method, include a job resource representation that contains:

   • Your location in the location property in the jobReference section.

   • The job ID generated by your client code. The server generates a job ID for you if you omit it, but it is a best practice to generate the job ID on the client side to allow reliable retry of the jobs.insert call.

     For example:

     {
       "jobReference": {
         "projectId": "my_project",
         "jobId": "job_123",
         “location”: “asia-northeast1”
       },
       "configuration":
       {
         // ..
       },
     }
     
     

2. In the configuration section of the job resource, include a child property that specifies the job type — load, query, extract, or copy.

3. After calling the jobs.insert method, check the job status by calling jobs.get with the job ID and location, and check the status.state value to learn the job status. When status.state is DONE, the job has stopped running; however, a DONE status does not mean that the job completed successfully, only that it is no longer running.

4. Check for job success. If the job has an errorResult property, the job has failed. The status.errorResult property holds information describing what went wrong in a failed job. If status.errorResult is absent, the job finished successfully, although there might have been some non-fatal errors, such as problems importing a few rows in a load job. Non-fatal errors are returned in the job's status.errors list.