talksik/genqlient
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Allow genqlient types to be marshaled safely (#120)

Browse Source 
## Summary:
When genqlient generates output types, it generates whatever code is
necessary to unmarshal them.  Conversely, when it generates input types,
it generates whatever code is necessary to marshal.  This is all that's
needed for genqlient itself: it never needs to marshal output types or
unmarshal input types.

But maybe you do!  (For example, to put the responses in a cache, which
is the use case that @csilvers hit at Khan, although there are others
one can imagine.)  While we can't support every serialization format you
might want (at least not without adding plugins or some such), it's not
unreasonable to expect that since genqlient can read JSON, it can write
it too.  Sadly, in the past this was not true for types requiring custom
unmarshaling logic, for several reasons.

In this commit I implement logic to always write both marshalers and
unmarshalers whenever they're needed to be able to correctly round-trip
the types, even though genqlient doesn't do so.  I wasn't starting from
scratch, since of course we already write both marshalers and
unmarshalers in some cases.  But this ended up requiring surprisingly
large changes on the marshaling side, mostly to correctly support
embedding (which we use for named fragments).

Specifically, as the comments in `types.go` discuss, the most difficult
issue is spreads with duplicate fields, which translate to Go embedded
fields which end up hidden from the json-marshaler.  Ultimately, I had
to do things quite differently from unmarshaling, and essentially
flatten the type when we write marshaler.  But in the end it's not so
ugly -- indeed arguably it's cleaner!  Mainly it's just different.

One thing to note is that we do marshal `__typename` based on
what we know about the types; users need not fill it in (and if they
do we'll ignore it).  This seemed to me to be a better UX, and
didn't add much complexity.

In general, I begin to wonder whether using `encoding/json` at all is
really right for genqlient: we're doing a lot of work to appease it,
despite knowing what our types look like.  I think it would still be a
significant increase in lines of code to roll our own, but that code
would perhaps be simpler, and would surely be faster (although if we
just want the speed gains we could use another JSON-generator library,
see also #47).  Anyway, something to think about in the future.

## Test plan:
make tesc


Author: benjaminjkraft

Reviewers: csilvers, StevenACoffman, benjaminjkraft, dnerdy, aberkan, jvoll, mahtabsabet, MiguelCastillo

Required Reviewers: 

Approved By: StevenACoffman, dnerdy

Checks: ✅ Test (1.17), ✅ Test (1.16), ✅ Test (1.15), ✅ Test (1.14), ✅ Lint, ✅ Test (1.17), ✅ Test (1.16), ✅ Test (1.15), ✅ Test (1.14), ✅ Lint

Pull Request URL: https://github.com/Khan/genqlient/pull/120
This commit is contained in:
Ben Kraft
2021-09-29 10:30:43 -07:00
committed by GitHub
parent 1f65445127
commit f4c981031e
30 changed files with 4143 additions and 180 deletions
Download Patch File Download Diff File Expand all files Collapse all files
generate/unmarshal.go.tmpl
+38 -17
View File
@@ -1,5 +1,23 @@
{{/* (the blank lines at the start are intentional, to separate
UnmarshalJSON from the function it follows) */}}
{{/* We need to generate UnmarshalJSON methods for some types that we want to
handle specially. Specifically, we generate an UnmarshalJSON for each
struct with a field meeting any of these criteria:
- a field whose type is configured with a custom unmarshaler
- an embedded (anonymous) field; technically we only need an
UnmarshalJSON if the embedded type needs one, but it's easier to
generate it unconditionally
- a field of interface type
Additionally, since we add `json:"-"` to fields we handle specially, for
any field which requires a MarshalJSON, we also generate an UnmarshalJSON,
and vice versa.
Given that, we want to specially handle the above-described fields, but
unmarshal everything else normally. To handle fields with custom
unmarshalers, first we unmarshal them into a json.RawMessage, then call
the custom unmarshaler. Interface-typed fields are similar, except
instead of a custom unmarshaler we call the helper we've generated
(see unmarshal_helper.go.tmpl). Embedded fields don't need the
json.RawMessage; we just unmarshal our input again into the embedded
field. */}}
func (v *{{.GoName}}) UnmarshalJSON(b []byte) error {
{{/* Standard convention for unmarshalers is to no-op on null. */}}
@@ -7,16 +25,14 @@ func (v *{{.GoName}}) UnmarshalJSON(b []byte) error {
return nil
}
{{/* We want to specially handle certain fields, but unmarshal everything
else normally. To handle abstract fields and fields with custom
unmarshalers, first we unmarshal them into a json.RawMessage, and then
handle those further, below. Embedded fields don't need the
json.RawMessage; we just use our input again. Either way, we first
want to call json.Unmarshal on the receiver (v). But if we do that
naively on a value of type `.Type`, it will call this function again,
and recurse infinitely. So we make a wrapper type which embeds both
this type and NoUmnarshalJSON, which prevents either's UnmarshalJSON
method from being promoted. For more on why this is so difficult, see
{{/* For our first pass, we ignore embedded fields, unmarshal all the
custom-unmarshaler or abstract fields into json.RawMessage, and
unmarshal everything else normally. To do this, we want to call
json.Unmarshal on the receiver (v). But if we do that naively on a
value of type <.GoName>, it will call this function again, and recurse
infinitely. So we make a wrapper type which embeds both this type and
NoUmnarshalJSON, which prevents either's UnmarshalJSON method from
being promoted. For more on why this is so difficult, see
https://github.com/benjaminjkraft/notes/blob/master/go-json-interfaces.md.
(Note there are a few different ways "hide" the method, but this one
seems to be the best option that works if this type has embedded types
@@ -28,7 +44,7 @@ func (v *{{.GoName}}) UnmarshalJSON(b []byte) error {
var firstPass struct{
*{{.GoName}}
{{range .Fields -}}
{{if and .NeedsUnmarshaler (not .IsEmbedded) -}}
{{if and .NeedsMarshaling (not .IsEmbedded) -}}
{{.GoName}} {{repeat .GoType.SliceDepth "[]"}}{{ref "encoding/json.RawMessage"}} `json:"{{.JSONName}}"`
{{end -}}
{{end -}}
@@ -45,11 +61,16 @@ func (v *{{.GoName}}) UnmarshalJSON(b []byte) error {
{{/* Now, handle the fields needing special handling. */}}
{{range $field := .Fields -}}
{{if $field.NeedsUnmarshaler -}}
{{if $field.NeedsMarshaling -}}
{{if $field.IsEmbedded -}}
{{/* Embedded fields are easier: we just unmarshal the same input into
them. (They're also easier because they can't be lists, since they
arise from GraphQL fragment spreads.) */ -}}
arise from GraphQL fragment spreads.)
Note that our behavior if you have two fields of the same name via
different embeds differs from ordinary json-unmarshaling: we unmarshal
into *all* of the fields. See goStructType.FlattenedFields in
types.go for more discussion of embedding and visibility. */ -}}
err = {{$field.Unmarshaler $.Generator}}(
b, &v.{{$field.GoType.Unwrap.Reference}})
if err != nil {
@@ -125,8 +146,8 @@ func (v *{{.GoName}}) UnmarshalJSON(b []byte) error {
}
{{end -}}
}
{{end -}}{{/* end if/else .IsEmbedded */ -}}
{{end -}}{{/* end if .NeedsUnmarshaler */ -}}
{{end}}{{/* end if/else .IsEmbedded */ -}}
{{end}}{{/* end if .NeedsMarshaling */ -}}
{{end}}{{/* end range .Fields */ -}}
return nil