3

I have installed CKAN v2.7.2(served under HTTPs using Apache web server),Nginx using docker. The task I am doing requires me to mount ckan on /ckan mount point, thus have added

ckan.root_path = /ckan/{{LANG}}

Also, I can access CKAN using Nginx only. CKAN is working properly using Nginx reverse proxy server,But there are three minor issues :

  1. I am unable to view private datasets successfully,whereas public datasets work perfectly fine.Whenever I login with admin account and try to view private datasets- it shows 404 Not Found error as below:

Private Dataset Preview

Additionally,in the preview section for private datasets,if I login through the same admin account as shown in yellow color(shown below) then I am able to preview the private datasets successfully.A different thing altogether!!

enter image description here After this the dataset gets previewed. I did try some changes in the configuration of Nginx but it didn't worked.Also,How can I view the Private datasets without logging IN as defined above,as I think this is not correct way of accessing CKAN using Nginx because whenever I clear the cache and to view private datasets,I always have to login in the preview section which I think is totally incorrect.

Update: 1

These were differences in request headers: On Left side- Requests, before I am unable to view the datasets Right Side - When I login in the preview section as shown above and view the private datasets successfully.

enter image description here

Update 2:

Sharing the Nginx default.conf file for CKAN :

      location /ckan {
            proxy_pass https://CKAN-IP:5000;

            proxy_set_header X-Url-Scheme $scheme;
            proxy_set_header Host $host;

            rewrite ^/ckan/ckan/(.*) /ckan/$1 permanent;

            proxy_cache cache_ckan;
            proxy_cache_bypass $cookie_auth_tkt;
            proxy_no_cache $cookie_auth_tkt;
            proxy_cache_valid 30m;
            proxy_cache_key $host$scheme$proxy_host$request_uri;

            limit_rate 25M;
    }

  1. Apache.wsgi script file

    import os
activate_this=os.path.join('/usr/lib/ckan/default/bin/activate_this.py')
execfile(activate_this, dict(__file__=activate_this))

from paste.deploy import loadapp
config_filepath=os.path.join(os.path.dirname(os.path.abspath(__file__)), 'ckan.ini')
from paste.script.util.logging_config import fileConfig
fileConfig(config_filepath)
_application = loadapp('config:%s' % config_filepath)

def application(environ, start_response):
  environ['wsgi.url_scheme']=environ.get('HTTP_X_URL_SCHEME', 'https')
  return _application(environ, start_response)

Moreover,I think that while viewing private datasets,my request goes from Nginx to Admin user,which is not logged in and hence is unable to find that resource which is private,thus displays Error 404 Not found above.

Update 3:

The Logs of Apache for CKAN are as follows:

 [Fri Jan 11 06:21:46.242529 2019] [wsgi:error] [pid 2178:tid 
 139975718889216] 2019-01-11 06:21:46,242 DEBUG [ckan.logic] check 
 access OK - dashboard_activity_list user=admin

 [Fri Jan 11 06:21:46.283912 2019] [wsgi:error] [pid 2178:tid 
 139975718889216] 2019-01-11 06:21:46,283 DEBUG [ckan.lib.base] rendering 
 /usr/lib/ckan/default/src/ckan/ckan/templates/snippets/home_breadcrumb_
 item.html[jinja2]

 [Fri Jan 11 06:21:46.285169 2019] [wsgi:error] [pid 2178:tid 
 139975718889216] 2019-01-11 06:21:46,285 DEBUG [ckan.lib.base] rendering 
 /usr/lib/ckan/default/src/ckan/ckan/templates/package/snippets/
 resources.html [jinja2]

 [Fri Jan 11 06:21:46.292565 2019] [wsgi:error] [pid 2178:tid 
 139975718889216] 2019-01-11 06:21:46,292 DEBUG [ckan.lib.base] rendering 
 /usr/lib/ckan/default/src/ckan/ckan/templates/snippets/social.html [jinja2]

 [Fri Jan 11 06:21:46.294569 2019] [wsgi:error] [pid 2178:tid 
 139975718889216] 2019-01-11 06:21:46,294 DEBUG [ckan.logic] check access OK 
 - package_update user=admin

 [Fri Jan 11 06:21:46.295155 2019] [wsgi:error] [pid 2178:tid 
 139975718889216] 2019-01-11 06:21:46,295 DEBUG [ckan.lib.base] rendering 
 /usr/lib/ckan/default/src/ckan/ckanext/datastore/templates/package/snippets
 /data_api_button.html [jinja2]

 [Fri Jan 11 06:21:46.299369 2019] [wsgi:error] [pid 2178:tid 
 139975718889216] 2019-01-11 06:21:46,299 DEBUG [ckan.lib.base] rendering 
 /usr/lib/ckan/default/src/ckan/ckan/templates/package/snippets/
 resource_views_list.html [jinja2]

 [Fri Jan 11 06:21:46.307521 2019] [wsgi:error] [pid 2178:tid 
 139975718889216] 2019-01-11 06:21:46,307 DEBUG [ckan.lib.base] rendering 
 /usr/lib/ckan/default/src/ckan/ckan/templates/package/snippets/
 resource_views_list_item.html [jinja2]

 [Fri Jan 11 06:21:46.314911 2019] [wsgi:error] [pid 2178:tid 
 139975718889216] 2019-01-11 06:21:46,314 DEBUG [ckan.lib.base] rendering 
 /usr/lib/ckan/default/src/ckan/ckan/templates/package/snippets/
 resource_view.html [jinja2]

 [Fri Jan 11 06:21:46.338195 2019] [wsgi:error] [pid 2178:tid 
 139975718889216] 2019-01-11 06:21:46,338 DEBUG [ckan.lib.base] rendering 
 /usr/lib/ckan/default/src/ckan/ckan/templates/snippets/license.html 
 [jinja2]

 [Fri Jan 11 06:21:46.339502 2019] [wsgi:error] [pid 2178:tid 
 139975718889216] 2019-01-11 06:21:46,339 DEBUG [ckan.lib.base] rendering 
 /usr/lib/ckan/default/src/ckan/ckan/templates/package/snippets/
 resources.html [jinja2]

 [Fri Jan 11 06:21:46.340241 2019] [wsgi:error] [pid 2178:tid 
 139975718889216] 2019-01-11 06:21:46,340 DEBUG [ckan.lib.base] rendering 
 /usr/lib/ckan/default/src/ckan/ckan/templates/snippets/social.html[jinja2]

 [Fri Jan 11 06:21:46.341363 2019] [wsgi:error] [pid 2178:tid 
 139975718889216] 2019-01-11 06:21:46,341 DEBUG [ckan.lib.base] rendering 
 usr/lib/ckan/default/src/ckan/ckan/templates/snippets/
 language_selector.html [jinja2]

 [Fri Jan 11 06:21:46.345452 2019] [wsgi:error] [pid 2178:tid 
 139975718889216] 2019-01-11 06:21:46,345 INFO  [ckan.lib.base]  
 /dataset/vfvfvfv/resource/1db088cc-2805-4c20-887c-f2f2ed91630e render time 
 0.209 seconds

 [Fri Jan 11 06:21:46.438541 2019] [wsgi:error] [pid 2178:tid 
 139975710496512] 2019-01-11 06:21:46,438 DEBUG 
 [ckan.config.middleware.pylons_app] Pylons route match: {'url': 
 u'base/images/nav-active.png', 'action': u'view', 'controller': 
 u'template'} Origin: core

 [Fri Jan 11 06:21:46.438661 2019] [wsgi:error] [pid 2178:tid 
 139975710496512] 2019-01-11 06:21:46,438 DEBUG [ckan.config.middleware] 
 Route support answers for GET /base/images/nav-active.png: [(False, 
 'flask_app'), (True, 'pylons_app', 'core')]

 [Fri Jan 11 06:21:46.438755 2019] [wsgi:error] [pid 2178:tid 
 139975710496512] 2019-01-11 06:21:46,438 DEBUG [ckan.config.middleware] 
 Serving request via pylons_app app

 [Fri Jan 11 06:21:46.447920 2019] [wsgi:error] [pid 2179:tid 
 139975718889216] 2019-01-11 06:21:46,447 DEBUG 
 [ckan.config.middleware.pylons_app] Pylons route match: {'lang': u'en', 
 'action': u'i18n_js_translations', 'controller': u'api', 'ver': u'/1'} 
 Origin: core

 [Fri Jan 11 06:21:46.448081 2019] [wsgi:error] [pid 2179:tid 
 139975718889216] 2019-01-11 06:21:46,448 DEBUG [ckan.config.middleware] 
 Route support answers for GET /api/i18n/en: [(False, 'flask_app'), (True, 
 'pylons_app', 'core')]

 [Fri Jan 11 06:21:46.448141 2019] [wsgi:error] [pid 2179:tid 
 139975718889216] 2019-01-11 06:21:46,448 DEBUG [ckan.config.middleware] 
 Serving request via pylons_app app

 **[Fri Jan 11 06:21:46.450241 2019] [wsgi:error] [pid 2179:tid 
 139975718889216] 2019-01-11 06:21:46,450 DEBUG [ckan.logic] check access OK 
 - site_read user=

 [Fri Jan 11 06:21:46.453059 2019] [wsgi:error] [pid 2179:tid 
 139975718889216] 2019-01-11 06:21:46,453 INFO  [ckan.lib.base]  
 /api/i18n/en render time 0.003 seconds

[Fri Jan 11 06:21:46.503467 2019] [wsgi:error] [pid 2178:tid 
139975718889216] 2019-01-11 06:21:46,503 DEBUG 
[ckan.config.middleware.pylons_app] Pylons route match: {'action': 
u'resource_view', 'view_id': u'a663fdda-ce6f-4ac2-b872-a646f488feff', 
'controller': u'package', 'id': u'vfvfvfv', 'resource_id': u'1db088cc-2805- 
4c20-887c-f2f2ed91630e'} Origin: core

[Fri Jan 11 06:21:46.503593 2019] [wsgi:error] [pid 2178:tid 
139975718889216] 2019-01-11 06:21:46,503 DEBUG [ckan.config.middleware] 
Route support answers for GET /dataset/vfvfvfv/resource/1db088cc-2805-4c20- 
887c-f2f2ed91630e/view/a663fdda-ce6f-4ac2-b872-a646f488feff: [(False, 
'flask_app'), (True, 'pylons_app', 'core')]

[Fri Jan 11 06:21:46.503652 2019] [wsgi:error] [pid 2178:tid 
139975718889216] 2019-01-11 06:21:46,503 DEBUG [ckan.config.middleware] 
Serving request via pylons_app app

**[Fri Jan 11 06:21:46.511546 2019] [wsgi:error] [pid 2178:tid 
139975718889216] 2019-01-11 06:21:46,511 DEBUG [ckan.logic] check access 
NotAuthorized - package_show user= "User  not authorized to read package 
1c10b0c0-2a2b-46e1-abc5-ff61f6794f8c"

I found the logs starting with ** the reason for not display of resource whereas in above logs at some places for check_access it shows admin.Only at this place there is no user(admin is a sysadmin user at ckan)

Improve this question
3
  • So this is for the resource View? Are there additional requests for the actual page because it seems you're already authenticated.
    – user3366016
    Jan 8, 2019 at 15:43
  • Yes,these are after I click preview of datasets.
    – argg
    Jan 9, 2019 at 9:14
  • 1
    Solved! The IP of nginx needs to be there in ckan.ini site_url parameter.
    – argg
    Feb 20, 2019 at 7:54

1 Answer 1

Reset to default
1

Well done on this issue / question. It's more succinct than before and focuses more closely on 1 problem. I'll focus on your 3 issues together as they seem to be interrelated.

You're right, I think you having to login in the preview section is totally incorrect and feels a bit odd :) and I can't seem to replicate this specific issue.

In your screenshot I can see the "Manage" button for the resource. This indicates on your CKAN instance you are logged in as an admin / editor of this dataset and should be able to view it and it's resources.

However, in the 404 message screenshot it's showing a "Log in" button/link. The whole embedded webpage thing feels very weird here. But it makes me think you're trying to link to something that is restricted or doesn't exist on another CKAN install (the demo site from the looks of things).

However, I tried to replicate this by looking at the URL shown for the resource in your screenshots and typing it in (painfulhttps://demo.ckan.org/ja/dataset/3a1d7942-44ff-48cd-b245-3636fe0a7e14/resource/fde84b28-cf0e-4748-b0fa-021ffbc88382/download/public_01.txt). And in my case a text view is not created and there isn't an option for one UNLESS I add resource_proxy to my production.ini`. Once I add the extension and upload the resource to the datastore my preview works as expected as long as I'm logged in to my CKAN instance. If I'm not logged in I get a 404 message as expected because it's a private dataset.

This makes me think it has to do with this issue and this note in the docs.

Suggested Fixes

You could:

  1. Make sure your private dataset resources are in the DataStore if you want to preview them and resource proxy is installed
  2. upload the file directly to your CKAN install, leaving it as a Private Dataset on your install if you desire
  3. link to the private dataset's resource on the demo.ckan.org install BUT remove the Text view and add a description for context for users and let them follow the URL link.
Improve this answer
4
  • All my data cannot be always in Datastore. However,my resource proxy is enabled. I am just not able to view the private datasets and if I change them to public it works fine.Do,I have to make some changes at github.com/ckan/ckan/blob/… ?
    – argg
    Jan 8, 2019 at 13:45
  • I'd lean towards no as this seems to be an issue somewhere on your end. From what I can tell I haven't seen others experience this issue specifically and when I've tested on my end I don't have the same issue. There's something odd to me about this page loading inside of a page and you authenticating twice.
    – user3366016
    Jan 8, 2019 at 15:39
  • That doesn't mean it isn't a bigger bug, I'm just leaning towards thinking it isn't.
    – user3366016
    Jan 8, 2019 at 15:56
  • Is it possible for you to share me with your nginx's default.conf file along with your .ini file if you've made any changes apart from ckan.root_path ? I shall try with your configuration and update you further.You can have a look at my conf files in Update 2 section.
    – argg
    Jan 8, 2019 at 18:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.