Pull GitHub repository using Bazel

Question

I need to download an entire GitHub repository using Bazel. Since I'm quite new to this tool I'm not really sure how to achieve that.

My main idea is this:

write a custom repository rule in downloadgithubrepo.bzl (which is located in the project root just like the WORKSPACE file) such as:

def _impl(repository_ctx):
    repository_ctx.download("url_to_zipped_github_repo", output='relative_path_to_output_file')

github = repository_rule(
    implementation = _impl

and in the WORKSPACE file to write something like this:

load("//:downloadgithubrepo.bzl", "github")

and to invoke a build a BUILD file is needed (also located at the project root) its contents are the following:

cc_library(
    name = "testrun",
    srcs = "main.c",
)

I had to add the main.c file otherwise the build is failing - that is one issue and the real issue is that this does not work, as in the build is passing but the GitHub repository is not downloaded.

Am I on the right path at all?? Has anyone done something like this before?

Answer 1

What you're looking might already be implemented in the new_git_repository repository rule, or the git_repository rule if the GitHub project already has Bazel BUILD files wired in.

If the GitHub project does not have BUILD files, a BUILD file is required when using new_git_repository. For example, if you want to depend on a file target (e.g. /foo/bar.txt) or rule target (e.g. a cc_library) in https://github.com/example/repository, and the repository does not have BUILD files, write these lines in your project's WORKSPACE file:

new_git_repository(
    name = "example_repository",
    remote = "https://github.com/example/repository.git",
    build_file_content = """
exports_files(["foo/bar.txt"])

# you can also create targets
cc_library(
    name = "remote_cc_library",
    srcs = ["..."],
    hdrs = ["..."],
)
""",
)

In your BUILD file, reference the external repository's targets using the @ prefix:

cc_library(
    name = "testrun",
    srcs = ["main.c"],
    data = ["@example_repository//:foo/bar.txt"],
    deps = ["@example_repository//:remote_cc_library"],
)

When you run bazel build //:testrun, Bazel will..

1. Analyze the dependencies of //:testrun, which include the file main.c and targets from the external repository @example_repository.
2. Look up the WORKSPACE file for an external repository named example_repository, and finds the new_git_repository declaration.
3. Perform a git clone on the remote attribute specified in the example_repository declaration.
4. Write a BUILD file containing the build_file_content string at the project root of the cloned repository.
5. Analyze the targets @example_repository//:foo/bar.txt and @example_repository//:remote_cc_library
6. Build the dependencies, and hands them to your //:testrun cc_library.
7. Build //:testrun.

If the GitHub project does have BUILD files, you do not need to provide an BUILD file. You can refer to the targets directly after specifying the WORKSPACE dependency with git_repository:

git_repository(
    name = "example_repository",
    remote = "https://github.com/example/repository.git",
)

For more information, check out Bazel's documentation on External Repositories.