Cannot unmarshal CloudEvent from pubsub push

[majelbstoat]

Sending a simple cloudevent message results in the following error:

Error: specversion: unknown : ""
, while converting event data: {"subscription":"projects\/my-project\/subscriptions\/my-subscription","message":{"data":"eyJ1c2VySWQiOiJBQUVCRlRpdXE0UFNEUEJ5UEEifQ==","messageId":"1","attributes":{"ce-source":"my-app","ce-specversion":"1.0","ce-type":"test.event.signIn","ce-id":"MFkWEgrOHKs44AEADQ","ce-time":"2020-05-24T18:38:18.7759848Z","Content-Type":"application\/json"}}}

Formatted event data for clarity:

{
  "subscription": "projects\/my-project\/subscriptions\/my-subscription",
  "message": {
    "data": "eyJ1c2VySWQiOiJBQUVCRlRpdXE0UFNEUEJ5UEEifQ==",
    "messageId": "1",
    "attributes": {
      "ce-source": "my-app",
      "ce-specversion": "1.0",
      "ce-type": "test.event.signIn",
      "ce-id": "MFkWEgrOHKs44AEADQ",
      "ce-time": "2020-05-24T18:38:18.7759848Z",
      "Content-Type": "application\/json"
    }
  }
}

[majelbstoat]

My function is:

package function

import (
	"context"
	"fmt"

	cloudevents "github.com/cloudevents/sdk-go/v2"
)

func CloudEventsFunction(ctx context.Context, e cloudevents.Event) error {
	fmt.Println("RECEIVED")
	return nil
}

and it is never called.

This is the library used to serialise the message:

https://github.com/cloudevents/sdk-go/blob/master/v2/protocol/pubsub/message.go

[majelbstoat]

Doing more digging, it appears that isStructuredCloudEvent expects cloud event data as headers, but they are instead present in the message attributes.

As a result, the framework attempts to parse the message as a legacyEvent. But, the fields in attributes are ce- prefixed, which I'm guessing is why they can't be parsed. (If it's parsed as structured, the ce- prefix is trimmed.)

The headers for the invocation are:

 <<<<< map[Accept:[text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2] Connection:[keep-alive] Content-Length:[360] Content-Type:[application/json] User-Agent:[Java/1.8.0_242]] >>>>>

I also tried sending a message with specversion 0.3, to see if it was serialised differently, but the same issue happens.

I also hacked the pubsub message writer in the sending library to not prefix the attributes:

https://github.com/cloudevents/sdk-go/blob/master/v2/protocol/pubsub/write_pubsub_message.go#L84

But that didn't work either:

Error: specversion: unknown : ""
, while converting event data: {"subscription":"projects\/my-project\/subscriptions\/my-subscription","message":{"data":"eyJ1c2VySWQiOiJBQUVCRlRpdXE0UFNEUEJ5UEEifQ==","messageId":"1","attributes":{"specversion":"1.0","time":"2020-05-24T20:29:48.2084708Z","id":"umYWEhDjne3NRAEADQ","source":"my-app","type":"test.event.signIn","Content-Type":"application\/json"}}}

[majelbstoat]

Well I guess here's the problem maybe? Functions framework uses the v1 of the cloud events.

cloudevents "github.com/cloudevents/sdk-go"

[tbpg]

Thank you for the report. #27 from @grant might be a duplicate of this.

[grant]

Hi @majelbstoat, a Pub/Sub push event does not deliver the event in CloudEvent format. It's not a CloudEvent, it's a custom JSON event.

As a result, you're seeing the unmarshal issue

Can you get this sample in the official docs to work?
https://cloud.google.com/functions/docs/calling/pubsub#sample_code

We're currently adding CloudEvent support to this repo.

[majelbstoat]

I ended up using the cloudevents SDK directly and creating a custom format to decode it, not entirely unlike the docs.

We're currently adding CloudEvent support to this repo.

Great!

[grant]

We have a PR out for CloudEvent support which will land shortly (#29)

We'll keep tracking CE support with #27 until it lands, so I'll go ahead and close this issue. We can continue the discussion there if that works. :)