Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How to setup OAuth2 #915

Closed
micol92 opened this issue Apr 21, 2024 · 13 comments
Closed

How to setup OAuth2 #915

micol92 opened this issue Apr 21, 2024 · 13 comments
Labels
bug Something isn't working

Comments

@micol92
Copy link

micol92 commented Apr 21, 2024

Describe the bug

A clear and concise description of what the bug is.

I'm trying to set up Answer in my SAP BTP Kyma runtime(aka managed K8s), confirmed that basic functions(Q&A) works well, and would like to specify OAuth2 configuation to connect my Identity Provider.

It looks that configuration tasks are almost done, but it fails after logging and erros occurrs as below.

---error message in my K8s Pod.
2024-04-21 13:50:00.316 ERROR connector-basic/basic.go:155 fail to get user id from json path: email

--error screen in my web page.
see the attached file.
apache_answer_error_log

To Reproduce

It's possible to reproduce, but if you really need to, we'll pass along the information you need to configure it.

Expected behavior

Can I modify the original source and leave a log for debugging?
Here is the source for the step before the error occurs, and I would like to see the data and userinfo.

----connector-basic/basic.go
data, _ := io.ReadAll(response.Body)

userInfo = plugin.ExternalLoginUserInfo{
	MetaInfo: string(data),
}

A clear and concise description of what you expected to happen.

Screenshots

see the attached file for error screen.

Platform

  • Device: Desktop
  • OS: K8S. my desktop : Mac book M1
  • Browser and version: Chrome, Safari
  • Version: �2.0
@micol92 micol92 added the bug Something isn't working label Apr 21, 2024
@LinkinStars
Copy link
Member

LinkinStars commented Apr 21, 2024

would like to specify OAuth2 configuation to connect my Identity Provider

@micol92

Is the OAuth2 implemented by yourself? If it is self-implemented, you need to return the JSON data in the API response, and the JSON should include a unique identifier, like user_id or something else.

If you have configured another OAuth2 service, please tell us which one you have configured. Google?

Additionally, providing your OAuth configuration can help us pinpoint the issue.

Can I modify the original source and leave a log for debugging?

You can build docker image with your own plugins. FYI: https://answer.apache.org/docs/plugins#build-docker-image-with-plugin-from-answer-base-image

Here is the source for the step before the error occurs, and I would like to see the data and userinfo.

Let me make it clear.

OAuth2 servier response JSON will be like.

{
    "user_id": "xxx",
    "eamil": "[email protected]"
}

The User ID JSON Path should config like user_id.

@micol92
Copy link
Author

micol92 commented Apr 21, 2024

Hello.
Thank you for your reply.

SAP BTP(Cloud Platform) provides IAS(Identity Authentication Service), and I used it for OAuth2 service.
For the OAuth2 configuration, see the attached file.

Apache_Answer_OAuth2_Basic

And OAuth2 response JSON is as followings. Please note what value I should use for User ID JSON Path.

{
  "firstname": "Jungwoo",
  "lastname": "Han",
  "email": "[email protected]",
  "name": "[email protected]",
  "scopes": [
    "openid",
    "uaa.user"
  ],
  "displayName": "Jungwoo Han ([email protected])"
}

For debugging the second source, I tried the following.
Before the source code where the error occurs, I added some source code to get some information.
And then, I tried to build using the guide link for source debugging and failed, so please make sure my Dockerfile is correct.

=====modified source codes in my github=========================

	log.Errorf("[jw log] data :  %s", data)
	
	userInfo = plugin.ExternalLoginUserInfo{
		MetaInfo: string(data),
	}
	log.Errorf("[jw log] userInfo :  %s", uerInfo)
	
	if len(g.Config.UserIDJsonPath) > 0 {
		userInfo.ExternalID = gjson.GetBytes(data, g.Config.UserIDJsonPath).String()
	}

=======================================================
=============Dockerfile for building Plugin functions=============
=======================================================

FROM apache/answer as answer-builder
ENV COREPACK_ENABLE_STRICT 0
FROM golang:1.19-alpine AS golang-builder

COPY --from=answer-builder /usr/bin/answer /usr/bin/answer

RUN apk --no-cache add \
    build-base git bash nodejs npm go && \
    npm install -g pnpm


RUN answer build --with github.com/micol92/jw-incubator-answer-plugins/connector-basic  --with github.com/apache/incubator-answer-plugins/storage-s3 --output /usr/bin/new_answer


FROM alpine
LABEL maintainer="[email protected]"

ARG TIMEZONE
ENV TIMEZONE=${TIMEZONE:-"Asia/Shanghai"}

RUN apk update \
    && apk --no-cache add \
        bash \
        ca-certificates \
        curl \
        dumb-init \
        gettext \
        openssh \
        sqlite \
        gnupg \
        tzdata \
    && ln -sf /usr/share/zoneinfo/${TIMEZONE} /etc/localtime \
    && echo "${TIMEZONE}" > /etc/timezone

COPY --from=golang-builder /usr/bin/new_answer /usr/bin/answer
COPY --from=answer-builder /data /data
COPY --from=answer-builder /entrypoint.sh /entrypoint.sh
RUN chmod 755 /entrypoint.sh

VOLUME /data
EXPOSE 80
ENTRYPOINT ["/entrypoint.sh"]

=======================================================
====================error log occured during building===================
=======================================================

go: finding module for package github.com/apache/incubator-answer-plugins/storage-s3
go: downloading github.com/micol92/jw-incubator-answer-plugins v1.2.1
go: downloading github.com/apache/incubator-answer-plugins v1.2.1
go: downloading github.com/micol92/jw-incubator-answer-plugins/connector-basic v1.2.6
go: downloading github.com/apache/incubator-answer v1.3.0
go: downloading github.com/apache/incubator-answer-plugins/storage-s3 v1.2.6
go: found github.com/apache/incubator-answer-plugins/storage-s3 in github.com/apache/incubator-answer-plugins/storage-s3 v1.2.6
go: found github.com/apache/incubator-answer/cmd in github.com/apache/incubator-answer v1.3.0
go: found github.com/micol92/jw-incubator-answer-plugins/connector-basic in github.com/micol92/jw-incubator-answer-plugins/connector-basic v1.2.6
go: answer imports
github.com/micol92/jw-incubator-answer-plugins/connector-basic: github.com/micol92/jw-incubator-answer-plugins/[email protected]: parsing go.mod:
module declares its path as: github.com/apache/incubator-answer-plugins/connector-basic
but was required as: github.com/micol92/jw-incubator-answer-plugins/connector-basic
build failed exit status 1

@LinkinStars
Copy link
Member

github.com/micol92/jw-incubator-answer-plugins/connector-basic: github.com/micol92/jw-incubator-answer-plugins/[email protected]: parsing go.mod:
module declares its path as: github.com/apache/incubator-answer-plugins/connector-basic

@micol92

Maybe you haven't modified the module declare to github.com/micol92/jw-incubator-answer-plugins/connector-basic in your go.mod file.

@micol92
Copy link
Author

micol92 commented Apr 21, 2024

Please check go.mod file modified by myself.
You can see the file in direct using this link.
https://github.com/micol92/jw-incubator-answer-plugins/blob/main/connector-basic/go.mod

================go.mod=============
module github.com/micol92/jw-incubator-answer-plugins/connector-basic

go 1.19

require (
	github.com/apache/incubator-answer v1.2.5
	github.com/segmentfault/pacman v1.0.5-0.20230822083413-c0075a2d401f
	github.com/tidwall/gjson v1.14.4
	golang.org/x/oauth2 v0.4.0
)

require (
	github.com/LinkinStars/go-i18n/v2 v2.2.2 // indirect
	github.com/Masterminds/semver/v3 v3.1.1 // indirect
	github.com/asaskevich/govalidator v0.0.0-20210307081110-f21760c49a8d // indirect
	github.com/aymerick/douceur v0.2.0 // indirect
	github.com/bytedance/sonic v1.9.1 // indirect
	github.com/chenzhuoyu/base64x v0.0.0-20221115062448-fe3a3abad311 // indirect
	github.com/dustin/go-humanize v1.0.1 // indirect
	github.com/gabriel-vasile/mimetype v1.4.2 // indirect
	github.com/gin-contrib/sse v0.1.0 // indirect
	github.com/gin-gonic/gin v1.9.1 // indirect
	github.com/go-playground/locales v0.14.1 // indirect
	github.com/go-playground/universal-translator v0.18.1 // indirect
	github.com/go-playground/validator/v10 v10.14.0 // indirect
	github.com/go-sql-driver/mysql v1.6.0 // indirect
	github.com/goccy/go-json v0.10.2 // indirect
	github.com/golang/protobuf v1.5.2 // indirect
	github.com/golang/snappy v0.0.4 // indirect
	github.com/google/uuid v1.3.0 // indirect
	github.com/google/wire v0.5.0 // indirect
	github.com/gorilla/css v1.0.0 // indirect
	github.com/json-iterator/go v1.1.12 // indirect
	github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51 // indirect
	github.com/klauspost/cpuid/v2 v2.2.4 // indirect
	github.com/leodido/go-urn v1.2.4 // indirect
	github.com/lib/pq v1.10.7 // indirect
	github.com/mattn/go-isatty v0.0.19 // indirect
	github.com/microcosm-cc/bluemonday v1.0.21 // indirect
	github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
	github.com/modern-go/reflect2 v1.0.2 // indirect
	github.com/patrickmn/go-cache v2.1.0+incompatible // indirect
	github.com/pelletier/go-toml/v2 v2.0.8 // indirect
	github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec // indirect
	github.com/segmentfault/pacman/contrib/cache/memory v0.0.0-20230822083413-c0075a2d401f // indirect
	github.com/segmentfault/pacman/contrib/i18n v0.0.0-20230516093754-b76aef1c1150 // indirect
	github.com/syndtr/goleveldb v1.0.0 // indirect
	github.com/tidwall/match v1.1.1 // indirect
	github.com/tidwall/pretty v1.2.0 // indirect
	github.com/twitchyliquid64/golang-asm v0.15.1 // indirect
	github.com/ugorji/go/codec v1.2.11 // indirect
	github.com/yuin/goldmark v1.4.13 // indirect
	golang.org/x/arch v0.3.0 // indirect
	golang.org/x/crypto v0.13.0 // indirect
	golang.org/x/image v0.1.0 // indirect
	golang.org/x/mod v0.12.0 // indirect
	golang.org/x/net v0.15.0 // indirect
	golang.org/x/sys v0.12.0 // indirect
	golang.org/x/text v0.13.0 // indirect
	golang.org/x/tools v0.13.0 // indirect
	google.golang.org/appengine v1.6.7 // indirect
	google.golang.org/protobuf v1.30.0 // indirect
	gopkg.in/yaml.v2 v2.4.0 // indirect
	gopkg.in/yaml.v3 v3.0.1 // indirect
	lukechampine.com/uint128 v1.2.0 // indirect
	modernc.org/cc/v3 v3.40.0 // indirect
	modernc.org/ccgo/v3 v3.16.13 // indirect
	modernc.org/libc v1.22.5 // indirect
	modernc.org/mathutil v1.5.0 // indirect
	modernc.org/memory v1.5.0 // indirect
	modernc.org/opt v0.1.3 // indirect
	modernc.org/sqlite v1.24.0 // indirect
	modernc.org/strutil v1.1.3 // indirect
	modernc.org/token v1.0.1 // indirect
	sigs.k8s.io/yaml v1.3.0 // indirect
	xorm.io/builder v0.3.12 // indirect
	xorm.io/xorm v1.3.2 // indirect
)

@LinkinStars
Copy link
Member

@micol92 OK, maybe I found the problem.

https://github.com/micol92/jw-incubator-answer-plugins/blob/50b0e6972341648e271a6169c5cb2ebaa9f9d77e/connector-basic/basic.go#L32C3-L32C66

You should import "github.com/micol92/jw-incubator-answer-plugins/connector-basic/i18n" not "github.com/apache/incubator-answer-plugins/connector-basic/i18n"

@LinkinStars
Copy link
Member

RUN apk --no-cache add build-base git bash nodejs npm go && npm install -g pnpm

BTW, using npm install -g [email protected] to avoid some of other problems.

@micol92
Copy link
Author

micol92 commented Apr 22, 2024

Thank you for your prompt update.

I've modified "basic.go" file you mentioned, but it failed. Please review the error log below.
And also ran the "npm install -g [email protected]".

By any chance, could you share a docker image that reflects the source code I'm curious about?
This is probably the best option for quick debugging.

============error log occurred while running docker build.
go: finding module for package github.com/apache/incubator-answer/cmd
go: downloading github.com/micol92/jw-incubator-answer-plugins/connector-basic v1.2.6
go: downloading github.com/apache/incubator-answer-plugins/storage-s3 v1.2.6
go: downloading github.com/micol92/jw-incubator-answer-plugins v1.2.1
go: downloading github.com/apache/incubator-answer-plugins v1.2.1
go: downloading github.com/apache/incubator-answer v1.3.0
go: found github.com/apache/incubator-answer-plugins/storage-s3 in github.com/apache/incubator-answer-plugins/storage-s3 v1.2.6
go: found github.com/apache/incubator-answer/cmd in github.com/apache/incubator-answer v1.3.0
go: found github.com/micol92/jw-incubator-answer-plugins/connector-basic in github.com/micol92/jw-incubator-answer-plugins/connector-basic v1.2.6
go: answer imports
github.com/micol92/jw-incubator-answer-plugins/connector-basic: github.com/micol92/jw-incubator-answer-plugins/[email protected]: parsing go.mod:
module declares its path as: github.com/apache/incubator-answer-plugins/connector-basic
but was required as: github.com/micol92/jw-incubator-answer-plugins/connector-basic
build failed exit status 1

@LinkinStars
Copy link
Member

@micol92 I make a PR to fix your compile error. micol92/jw-incubator-answer-plugins#1

After merge it, add new tag and build with new tag version.

$ git pull
$ git tag connector-basic/v1.2.7
$ git push origin connector-basic/v1.2.7
FROM apache/answer as answer-builder

FROM golang:1.19-alpine AS golang-builder

COPY --from=answer-builder /usr/bin/answer /usr/bin/answer

RUN apk --no-cache add \
    build-base git bash nodejs npm go && \
    npm install -g [email protected]

RUN answer build \
    --with github.com/micol92/jw-incubator-answer-plugins/[email protected] \
    --output /usr/bin/new_answer

FROM alpine
LABEL maintainer="[email protected]"

ARG TIMEZONE
ENV TIMEZONE=${TIMEZONE:-"Asia/Shanghai"}

RUN apk update \
    && apk --no-cache add \
        bash \
        ca-certificates \
        curl \
        dumb-init \
        gettext \
        openssh \
        sqlite \
        gnupg \
        tzdata \
    && ln -sf /usr/share/zoneinfo/${TIMEZONE} /etc/localtime \
    && echo "${TIMEZONE}" > /etc/timezone

COPY --from=golang-builder /usr/bin/new_answer /usr/bin/answer
COPY --from=answer-builder /data /data
COPY --from=answer-builder /entrypoint.sh /entrypoint.sh
RUN chmod 755 /entrypoint.sh

VOLUME /data
EXPOSE 80
ENTRYPOINT ["/entrypoint.sh"]

@micol92
Copy link
Author

micol92 commented Apr 23, 2024

I'm very appreciate with your help.
The added logs gave me a hint and I fixed the issue: the root cause was an invalid User JSON URL.
Next, I would like to change database setting from SQLite to PostgreSQL, and also apply AWS S3 and ElasticSearch plug-ins.
One last question is that Do I have to redeploy docker image to change database setting ? I could not find out the menu to change database setting after deployment.
Once I have your answer, let me close this issue. Thanks again.

@micol92
Copy link
Author

micol92 commented Apr 23, 2024

As an additional question, if I build the AWS S3 plugin, will I be able to add attachments to my questions? And if I add the Elastic search plugin, will I be able to perform full-text search on attachments?

@LinkinStars
Copy link
Member

I'm very appreciate with your help. The added logs gave me a hint and I fixed the issue: the root cause was an invalid User JSON URL. Next, I would like to change database setting from SQLite to PostgreSQL, and also apply AWS S3 and ElasticSearch plug-ins. One last question is that Do I have to redeploy docker image to change database setting ? I could not find out the menu to change database setting after deployment. Once I have your answer, let me close this issue. Thanks again.

@micol92 You need redeploy again. Becasue we do not support data migration currently. FYI: https://answer.apache.org/docs/faq#what-are-the-currently-supported-database-types-is-migration-supported

@LinkinStars
Copy link
Member

As an additional question, if I build the AWS S3 plugin, will I be able to add attachments to my questions? And if I add the Elastic search plugin, will I be able to perform full-text search on attachments?

@micol92 S3 only save the images. FYI: #211

@micol92
Copy link
Author

micol92 commented Apr 24, 2024

Thank you.

@micol92 micol92 closed this as completed Apr 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants