Update Resource URL automatically when replicating CKAN to another AWS environment via CloudFront?

0 votes

We are implementing CKAN on AWS with the DataStore extension and interact with it via the python CKAN API. AWS is essentially split into two environments:

PUB is created via CloudFront and uses Read Replica of the PRIV database. It is basically the same as PRIV except that it is read-only.

Challenge

Resource URLs in PUB point to the PRIV environment. For example, running PUB_ckan.resource_show(id='123') api call in the public environment returns the following:

{ ...

'datastore_active': False,

'id': '123',

'name': 'Resource 1',

'package_id': 'abc',

'state': 'active',

'url': 'https://private.company.com/dataset/f688/resource/e3c785/download/file.zip',

'url_type': 'upload'

... }

This is the same for files uploaded through the CKAN API or the DataStore extension (in which case they are labeled 'url_type': 'datastore).

Expectation

All of the package/resource metadata should be the same between environments, with the exception of Resource URL which must reflect the PUB URL so end-users make api calls against the highly available, secure, environment. i.e.:

'url': 'https://public.company.com/dataset/f688/resource/e3c785/download/file.zip'

So far, I have looked into whether the config file contains a setting for using relative URLs and also tried updating the URLs manually via a Python script, both without success. Any help with this would be greatly appreciated.

Aug 21, 2018 in AWS by bug_seeker
• 15,350 points
52 views

1 answer to this question.

0 votes

If you want to keep using the same underlying database then the easiest way is probably a small plugin on the public instance which implements the IResourceController interface and uses the before_show method of that interface to change the resource URL.

Note, however, the following warning from the documentation of before_show:

Be aware that this method is not only called for UI display, but also in other methods like when a resource is deleted because showing a package is used to get access to the resources in a package.

You should therefore definitely test that your URL modifications do not affect other parts of CKAN.

answered Aug 21, 2018 by Priyaj
• 56,520 points

Related Questions In AWS

0 votes
1 answer

Trying to call AWS API via PHP

Try using AWS SDK for PHP, Link ...READ MORE

answered Jun 6, 2018 in AWS by Cloud gunner
• 4,260 points
227 views
0 votes
1 answer

How to Enable CORS for an AWS API Gateway Resource

Since you're using Lambda Proxy integration for ...READ MORE

answered Jul 10, 2018 in AWS by Hammer
• 360 points
637 views
0 votes
1 answer
0 votes
1 answer
0 votes
1 answer