CKAN: Harvest special data sets

Question

I want to import a special set of data sets in my CKAN instance. With the CKAN-Harvester(http://docs.ckan.org/en/latest/harvesting.html) i am able to harvest from an other CKAN instance. But i don't need all of the data sets.

Is it possible to harvest only specific data sets with their id?

Answer 1

Not without writing some code.

You could add a filter to the harvester. gather_stage() [1] is where it asks CKAN for the latest edited packages (datasets) and creates a job for each one. Then the fetch_stage() [2] runs for each of those jobs to download each one and then import them. You might a filter in the fetch_stage, or alternatively change the gather_stage to ask for a subset of packages.

[1] https://github.com/okfn/ckanext-harvest/blob/2.0-dataset-sources/ckanext/harvest/harvesters/ckanharvester.py#L136

[2] https://github.com/okfn/ckanext-harvest/blob/2.0-dataset-sources/ckanext/harvest/harvesters/ckanharvester.py#L199