Document and improve support for binding non-scalars to a specific type (#69)

## Summary:
We had this setting called "scalars", which said: bind this GraphQL type
to this Go type, rather than the one you would normally use.  It's
called that because it's most useful for custom scalars, where "the one
you would normally use" is "error: unknown scalar".  But nothing ever
stopped you from using it for a non-scalar type.  I was planning on
removing this functionality, because it's sort of a rough edge, but a
discussion with Craig found some good use cases, so instead, in this
commit, I document it better and add some slightly nicer ways to specify
it.

Specifically, here are a few potential non-scalar use cases:
- bind a GraphQL enum to a nonstandard type (or even `string`)
- bind an input type to some type that has exactly the fields you want;
  this acts as a sort of workaround for issues #14 and #44
- bind an object type to your own struct, so as to add methods to it
  (this is the use case Craig raised)
- bind an object type to your own struct, so as to share it between
  multiple queries (I believe named fragments will address this case
  better, but it doesn't hurt to have options)
- bind a GraphQL list type to a non-slice type in Go (presumably one
  with an UnmarshalJSON method), or any other different structure
The latter three cases still have the sharp edge I was originally
worried about, which is that nothing guarantees that the fields you
request in the query are the ones the type expects to get.  But I think
it's worth having the option, with appropriate disclaimers.

The main change to help support that better is that you can now specify
the type inline in the query, as an alternative to specifying it in the
config file; this means you might map a given object to a given struct,
but only in some cases, and when you do you have a chance to look at the
list of fields you're requesting.

Additionally, I renamed the config field from "scalars" to "bindings"
(but mentioned it in a few places where you might go looking for how to
map scalars, most importantly the error message you get for an unknown
(custom) scalar).  While I was making a breaking change, I also changed
it to be a `map[string]<struct>` instead of a `map[string]string`,
because I expect to add more fields soon, e.g. to handle issue #38.

Finally, since the feature is now intended/documented, I added some
tests, although it's honestly quite simple on the genqlient side.

## Test plan:
make tesc


Author: benjaminjkraft

Reviewers: csilvers, aberkan, dnerdy, MiguelCastillo

Required Reviewers: 

Approved by: csilvers

Checks: ⌛ Test (1.17), ⌛ Test (1.16), ⌛ Test (1.15), ⌛ Test (1.14), ⌛ Test (1.13), ⌛ Lint, ⌛ Test (1.17), ⌛ Test (1.16), ⌛ Test (1.15), ⌛ Test (1.14), ⌛ Test (1.13), ⌛ Lint

Pull request URL: https://github.com/Khan/genqlient/pull/69

example/genqlient.yaml

@@ -5,7 +5,10 @@ operations:
 - genqlient.graphql
 generated: generated.go
 
-# We map github's DateTime type to Go's time.Time (which conveniently already
-# defines MarshalJSON and UnmarshalJSON).
-scalars:
-  DateTime: time.Time
+# We bind github's DateTime scalar type to Go's time.Time (which conveniently
+# already defines MarshalJSON and UnmarshalJSON).  This means genqlient will
+# use time.Time when a query requests a DateTime, and is required for custom
+# scalars.
+bindings:
+  DateTime:
+    type: time.Time

generate/config.go

@@ -65,13 +65,25 @@ type Config struct {
 	// TODO(#5): This is a bit broken, fix it.
 	ClientGetter string `yaml:"client_getter"`
 
-	// A map from GraphQL scalar type name to Go fully-qualified type name for
-	// the types to use for any custom or builtin scalars.  By default, builtin
-	// scalars are mapped to the obvious Go types (String and ID to string, Int
-	// to int, Float to float64, and Boolean to bool), but this setting will
-	// extend or override those mappings.  These types must define MarshalJSON
-	// and UnmarshalJSON methods, or otherwise be convertible to JSON.
-	Scalars map[string]string `yaml:"scalars"`
+	// A map from GraphQL type name to Go fully-qualified type name to override
+	// the Go type genqlient will use for this GraphQL type.
+	//
+	// This is primarily used for custom scalars, or to map builtin scalars to
+	// a nonstandard type.  By default, builtin scalars are mapped to the
+	// obvious Go types (String and ID to string, Int to int, Float to float64,
+	// and Boolean to bool), but this setting will extend or override those
+	// mappings.
+	//
+	// genqlient does not validate these types in any way; they must define
+	// whatever logic is needed (MarshalJSON/UnmarshalJSON or JSON tags) to
+	// convert to/from JSON.  For this reason, it's not recommended to use this
+	// setting to map object, interface, or union types, because nothing
+	// guarantees that the fields requested in the query match those present in
+	// the Go type.
+	//
+	// To get equivalent behavior in just one query, use @genqlient(bind: ...);
+	// see GenqlientDirective.Bind for more details.
+	Bindings map[string]*TypeBinding `yaml:"bindings"`
 
 	// Set to true to use features that aren't fully ready to use.
 	//
@@ -84,6 +96,16 @@ type Config struct {
 	configFilename string
 }
 
+// A TypeBinding represents a Go type to which genqlient will bind a particular
+// GraphQL type.  See Config.Bind, above, for more details.
+type TypeBinding struct {
+	// The fully-qualified name of the Go type to which to bind.  For example:
+	//	time.Time
+	//  map[string]interface{}
+	//	github.com/you/yourpkg/subpkg.MyType
+	Type string `yaml:"type"`
+}
+
 // baseDir returns the directory of the config-file (relative to which
 // all the other paths are resolved).
 func (c *Config) baseDir() string {

generate/convert.go

@@ -53,7 +53,7 @@ func (g *generator) convertOperation(
 
 	goTyp, err := g.convertDefinition(
 		name, operation.Name, baseType, operation.Position,
-		operation.SelectionSet, queryOptions)
+		operation.SelectionSet, queryOptions, queryOptions)
 
 	if structType, ok := goTyp.(*goStructType); ok {
 		// Override the ordinary description; the GraphQL documentation for
@@ -140,6 +140,15 @@ func (g *generator) convertType(
 	selectionSet ast.SelectionSet,
 	options, queryOptions *GenqlientDirective,
 ) (goType, error) {
+	// We check for local bindings here, so that you can bind, say, a
+	// `[String!]` to a struct instead of a slice.  Global bindings can only
+	// bind GraphQL named types, at least for now.
+	localBinding := options.Bind
+	if localBinding != "" && localBinding != "-" {
+		goRef, err := g.addRef(localBinding)
+		return &goOpaqueType{goRef}, err
+	}
+
 	if typ.Elem != nil {
 		// Type is a list.
 		elem, err := g.convertType(
@@ -150,7 +159,7 @@ func (g *generator) convertType(
 	// If this is a builtin type or custom scalar, just refer to it.
 	def := g.schema.Types[typ.Name()]
 	goTyp, err := g.convertDefinition(
-		name, namePrefix, def, typ.Position, selectionSet, queryOptions)
+		name, namePrefix, def, typ.Position, selectionSet, options, queryOptions)
 
 	if options.GetPointer() {
 		// Whatever we get, wrap it in a pointer.  (Because of the way the
@@ -172,11 +181,16 @@ func (g *generator) convertDefinition(
 	def *ast.Definition,
 	pos *ast.Position,
 	selectionSet ast.SelectionSet,
-	queryOptions *GenqlientDirective,
+	options, queryOptions *GenqlientDirective,
 ) (goType, error) {
-	qualifiedGoName, ok := g.Config.Scalars[def.Name]
-	if ok {
-		goRef, err := g.addRef(qualifiedGoName)
+	// Check if we should use an existing type.  (This is usually true for
+	// GraphQL scalars, but we allow you to bind non-scalar types too, if you
+	// want, subject to the caveats described in Config.Bindings.)  Local
+	// bindings are checked in the caller (convertType) and never get here,
+	// unless the binding is "-" which means "ignore the global binding".
+	globalBinding, ok := g.Config.Bindings[def.Name]
+	if ok && options.Bind != "-" {
+		goRef, err := g.addRef(globalBinding.Type)
 		return &goOpaqueType{goRef}, err
 	}
 	goBuiltinName, ok := builtinTypes[def.Name]
@@ -306,7 +320,7 @@ func (g *generator) convertDefinition(
 			// preprocessQueryDocument).  But in practice it doesn't really
 			// hurt, and would be extra work to avoid, so we just leave it.
 			implTyp, err := g.convertDefinition(
-				implName, namePrefix, implDef, pos, selectionSet, queryOptions)
+				implName, namePrefix, implDef, pos, selectionSet, options, queryOptions)
 			if err != nil {
 				return nil, err
 			}
@@ -335,8 +349,9 @@ func (g *generator) convertDefinition(
 		return goType, nil
 
 	case ast.Scalar:
+		// (If you had an entry in bindings, we would have returned it above.)
 		return nil, errorf(
-			pos, "unknown scalar %v: please add it to genqlient.yaml", def.Name)
+			pos, `unknown scalar %v: please add it to "bindings" in genqlient.yaml`, def.Name)
 	default:
 		return nil, errorf(pos, "unexpected kind: %v", def.Kind)
 	}

generate/generate_test.go

@@ -51,11 +51,13 @@ func TestGenerate(t *testing.T) {
 				Package:          "test",
 				Generated:        goFilename,
 				ExportOperations: queriesFilename,
-				Scalars: map[string]string{
-					"ID":          "github.com/Khan/genqlient/internal/testutil.ID",
-					"DateTime":    "time.Time",
-					"Junk":        "interface{}",
-					"ComplexJunk": "[]map[string]*[]*map[string]interface{}",
+				Bindings: map[string]*TypeBinding{
+					"ID":           {Type: "github.com/Khan/genqlient/internal/testutil.ID"},
+					"DateTime":     {Type: "time.Time"},
+					"Junk":         {Type: "interface{}"},
+					"ComplexJunk":  {Type: "[]map[string]*[]*map[string]interface{}"},
+					"Pokemon":      {Type: "github.com/Khan/genqlient/internal/testutil.Pokemon"},
+					"PokemonInput": {Type: "github.com/Khan/genqlient/internal/testutil.Pokemon"},
 				},
 				AllowBrokenFeatures: true,
 			})
@@ -138,9 +140,9 @@ func TestGenerateErrors(t *testing.T) {
 				Operations: []string{filepath.Join(errorsDir, sourceFilename)},
 				Package:    "test",
 				Generated:  os.DevNull,
-				Scalars: map[string]string{
-					"ValidScalar":   "string",
-					"InvalidScalar": "bogus",
+				Bindings: map[string]*TypeBinding{
+					"ValidScalar":   {Type: "string"},
+					"InvalidScalar": {Type: "bogus"},
 				},
 				AllowBrokenFeatures: true,
 			})

generate/genqlient_directive.go

@@ -63,6 +63,27 @@ type GenqlientDirective struct {
 	// pointer to save copies) or if you wish to distinguish between the Go
 	// zero value and null (for nullable fields).
 	Pointer *bool
+
+	// If set, this argument or field will use the given Go type instead of a
+	// genqlient-generated type.
+	//
+	// The value should be the fully-qualified type name to use for the field,
+	// for example:
+	//	time.Time
+	//	map[string]interface{}
+	//	[]github.com/you/yourpkg/subpkg.MyType
+	// Note that the type is the type of the whole field, e.g. if your field in
+	// GraphQL has type `[DateTime]`, you'd do
+	//	# @genqlient(bind: "[]time.Time")
+	// (But you're not required to; if you want to map to some type DateList,
+	// you can do that, as long as its UnmarshalJSON method can accept a list
+	// of datetimes.)
+	//
+	// See Config.Bindings for more details; this is effectively to a local
+	// version of that global setting and should be used with similar care.
+	// If set to "-", overrides any such global setting and uses a
+	// genqlient-generated type.
+	Bind string
 }
 
 func (dir *GenqlientDirective) GetOmitempty() bool { return dir.Omitempty != nil && *dir.Omitempty }
@@ -70,7 +91,6 @@ func (dir *GenqlientDirective) GetPointer() bool   { return dir.Pointer != nil &
 
 func setBool(dst **bool, v *ast.Value) error {
 	ei, err := v.Value(nil) // no vars allowed
-	// TODO: here and below, put positions on these errors
 	if err != nil {
 		return errorf(v.Position, "invalid boolean value %v: %v", v, err)
 	}
@@ -81,6 +101,18 @@ func setBool(dst **bool, v *ast.Value) error {
 	return errorf(v.Position, "expected boolean, got non-boolean value %T(%v)", ei, ei)
 }
 
+func setString(dst *string, v *ast.Value) error {
+	ei, err := v.Value(nil) // no vars allowed
+	if err != nil {
+		return errorf(v.Position, "invalid string value %v: %v", v, err)
+	}
+	if b, ok := ei.(string); ok {
+		*dst = b
+		return nil
+	}
+	return errorf(v.Position, "expected string, got non-string value %T(%v)", ei, ei)
+}
+
 func fromGraphQL(dir *ast.Directive) (*GenqlientDirective, error) {
 	if dir.Name != "genqlient" {
 		// Actually we just won't get here; we only get here if the line starts
@@ -99,6 +131,8 @@ func fromGraphQL(dir *ast.Directive) (*GenqlientDirective, error) {
 			err = setBool(&retval.Omitempty, arg.Value)
 		case "pointer":
 			err = setBool(&retval.Pointer, arg.Value)
+		case "bind":
+			err = setString(&retval.Bind, arg.Value)
 		default:
 			return nil, errorf(arg.Position, "unknown argument %v for @genqlient", arg.Name)
 		}
@@ -112,8 +146,12 @@ func fromGraphQL(dir *ast.Directive) (*GenqlientDirective, error) {
 func (dir *GenqlientDirective) validate(node interface{}) error {
 	switch node := node.(type) {
 	case *ast.OperationDefinition:
-		// Anything is valid on the entire operation; it will just apply to
-		// whatever it is relevant to.
+		if dir.Bind != "" {
+			return errorf(dir.pos, "bind may not be applied to the entire operation")
+		}
+
+		// Anything else is valid on the entire operation; it will just apply
+		// to whatever it is relevant to.
 		return nil
 	case *ast.VariableDefinition:
 		if dir.Omitempty != nil && node.Type.NonNull {
@@ -122,7 +160,7 @@ func (dir *GenqlientDirective) validate(node interface{}) error {
 		return nil
 	case *ast.Field:
 		if dir.Omitempty != nil {
-			return errorf(dir.pos, "omitempty is not appilcable to fields")
+			return errorf(dir.pos, "omitempty is not applicable to fields")
 		}
 		return nil
 	default:
@@ -138,6 +176,9 @@ func (dir *GenqlientDirective) merge(other *GenqlientDirective) *GenqlientDirect
 	if other.Pointer != nil {
 		retval.Pointer = other.Pointer
 	}
+	if other.Bind != "" {
+		retval.Bind = other.Bind
+	}
 	return &retval
 }
 

generate/testdata/queries/Pokemon.graphql

@@ -0,0 +1,16 @@
+query GetPokemonSiblings($input: PokemonInput!) {
+  user(query: {hasPokemon: $input}) {
+    # this will override the default mapping to internal/testutil.ID:
+    # @genqlient(bind: "string")
+    id
+    # this is normally an enum, but here we make it a (list of) string:
+    # @genqlient(bind: "[]string")
+    roles
+    name
+    # this is mapped globally to internal/testutil.Pokemon:
+    pokemon { species level }
+    # this overrides said mapping:
+    # @genqlient(bind: "-")
+    genqlientPokemon: pokemon { species level }
+  }
+}

generate/testdata/queries/schema.graphql

@@ -20,6 +20,16 @@ enum Role {
   TEACHER
 }
 
+input PokemonInput {
+  species: String!
+  level: Int!
+}
+
+type Pokemon {
+  species: String!
+  level: Int!
+}
+
 """UserQueryInput is the argument to Query.users.
 
 Ideally this would support anything and everything!
@@ -33,6 +43,7 @@ input UserQueryInput {
   id: ID
   role: Role
   names: [String]
+  hasPokemon: PokemonInput
 }
 
 type AuthMethod {
@@ -53,6 +64,7 @@ type User {
   emailsWithNulls: [String]!
   emailsWithNullsOrNull: [String]
   authMethods: [AuthMethod!]!
+  pokemon: [Pokemon!]
 }
 
 """Content is implemented by various types like Article, Video, and Topic."""

generate/testdata/snapshots/TestGenerate-InputObject.graphql-InputObject.graphql.go

@@ -50,9 +50,10 @@ type UserQueryInput struct {
 	Email string `json:"email"`
 	Name  string `json:"name"`
 	// id looks the user up by ID.  It's a great way to look up users.
-	Id    testutil.ID `json:"id"`
-	Role  Role        `json:"role"`
-	Names []string    `json:"names"`
+	Id         testutil.ID      `json:"id"`
+	Role       Role             `json:"role"`
+	Names      []string         `json:"names"`
+	HasPokemon testutil.Pokemon `json:"hasPokemon"`
 }
 
 func InputObjectQuery(

generate/testdata/snapshots/TestGenerate-Omitempty.graphql-Omitempty.graphql.go

@@ -66,9 +66,10 @@ type UserQueryInput struct {
 	Email string `json:"email"`
 	Name  string `json:"name"`
 	// id looks the user up by ID.  It's a great way to look up users.
-	Id    testutil.ID `json:"id"`
-	Role  Role        `json:"role"`
-	Names []string    `json:"names"`
+	Id         testutil.ID      `json:"id"`
+	Role       Role             `json:"role"`
+	Names      []string         `json:"names"`
+	HasPokemon testutil.Pokemon `json:"hasPokemon"`
 }
 
 func OmitEmptyQuery(

generate/testdata/snapshots/TestGenerate-Pointers.graphql-Pointers.graphql.go

@@ -73,9 +73,10 @@ type UserQueryInput struct {
 	Email *string `json:"email"`
 	Name  *string `json:"name"`
 	// id looks the user up by ID.  It's a great way to look up users.
-	Id    *testutil.ID `json:"id"`
-	Role  *Role        `json:"role"`
-	Names []*string    `json:"names"`
+	Id         *testutil.ID      `json:"id"`
+	Role       *Role             `json:"role"`
+	Names      []*string         `json:"names"`
+	HasPokemon *testutil.Pokemon `json:"hasPokemon"`
 }
 
 func PointersQuery(

generate/testdata/snapshots/TestGenerate-PointersInline.graphql-PointersInline.graphql.go

@@ -73,9 +73,10 @@ type UserQueryInput struct {
 	Email string `json:"email"`
 	Name  string `json:"name"`
 	// id looks the user up by ID.  It's a great way to look up users.
-	Id    testutil.ID `json:"id"`
-	Role  Role        `json:"role"`
-	Names []string    `json:"names"`
+	Id         testutil.ID      `json:"id"`
+	Role       Role             `json:"role"`
+	Names      []string         `json:"names"`
+	HasPokemon testutil.Pokemon `json:"hasPokemon"`
 }
 
 func PointersQuery(

generate/testdata/snapshots/TestGenerate-Pokemon.graphql-Pokemon.graphql.go

@@ -0,0 +1,74 @@
+package test
+
+// Code generated by github.com/Khan/genqlient, DO NOT EDIT.
+
+import (
+	"github.com/Khan/genqlient/graphql"
+	"github.com/Khan/genqlient/internal/testutil"
+)
+
+// GetPokemonSiblingsResponse is returned by GetPokemonSiblings on success.
+type GetPokemonSiblingsResponse struct {
+	// user looks up a user by some stuff.
+	//
+	// See UserQueryInput for what stuff is supported.
+	// If query is null, returns the current user.
+	User GetPokemonSiblingsUser `json:"user"`
+}
+
+// GetPokemonSiblingsUser includes the requested fields of the GraphQL type User.
+// The GraphQL type's documentation follows.
+//
+// A User is a user!
+type GetPokemonSiblingsUser struct {
+	// id is the user's ID.
+	//
+	// It is stable, unique, and opaque, like all good IDs.
+	Id               string                                   `json:"id"`
+	Roles            []string                                 `json:"roles"`
+	Name             string                                   `json:"name"`
+	Pokemon          []testutil.Pokemon                       `json:"pokemon"`
+	GenqlientPokemon []GetPokemonSiblingsUserGenqlientPokemon `json:"genqlientPokemon"`
+}
+
+// GetPokemonSiblingsUserGenqlientPokemon includes the requested fields of the GraphQL type Pokemon.
+type GetPokemonSiblingsUserGenqlientPokemon struct {
+	Species string `json:"species"`
+	Level   int    `json:"level"`
+}
+
+func GetPokemonSiblings(
+	client graphql.Client,
+	input testutil.Pokemon,
+) (*GetPokemonSiblingsResponse, error) {
+	variables := map[string]interface{}{
+		"input": input,
+	}
+
+	var retval GetPokemonSiblingsResponse
+	err := client.MakeRequest(
+		nil,
+		"GetPokemonSiblings",
+		`
+query GetPokemonSiblings ($input: PokemonInput!) {
+	user(query: {hasPokemon:$input}) {
+		id
+		roles
+		name
+		pokemon {
+			species
+			level
+		}
+		genqlientPokemon: pokemon {
+			species
+			level
+		}
+	}
+}
+`,
+		&retval,
+		variables,
+	)
+	return &retval, err
+}
+

generate/testdata/snapshots/TestGenerate-Pokemon.graphql-Pokemon.graphql.json

@@ -0,0 +1,9 @@
+{
+  "operations": [
+    {
+      "operationName": "GetPokemonSiblings",
+      "query": "\nquery GetPokemonSiblings ($input: PokemonInput!) {\n\tuser(query: {hasPokemon:$input}) {\n\t\tid\n\t\troles\n\t\tname\n\t\tpokemon {\n\t\t\tspecies\n\t\t\tlevel\n\t\t}\n\t\tgenqlientPokemon: pokemon {\n\t\t\tspecies\n\t\t\tlevel\n\t\t}\n\t}\n}\n",
+      "sourceLocation": "testdata/queries/Pokemon.graphql"
+    }
+  ]
+}

generate/testdata/snapshots/TestGenerate-unexported.graphql-unexported.graphql.go

@@ -50,9 +50,10 @@ type userQueryInput struct {
 	Email string `json:"email"`
 	Name  string `json:"name"`
 	// id looks the user up by ID.  It's a great way to look up users.
-	Id    testutil.ID `json:"id"`
-	Role  Role        `json:"role"`
-	Names []string    `json:"names"`
+	Id         testutil.ID      `json:"id"`
+	Role       Role             `json:"role"`
+	Names      []string         `json:"names"`
+	HasPokemon testutil.Pokemon `json:"hasPokemon"`
 }
 
 func unexported(

generate/testdata/snapshots/TestGenerateErrors-UnknownScalar.go

@@ -1 +1 @@
-testdata/errors/UnknownScalar.schema.graphql:3: unknown scalar UnknownScalar: please add it to genqlient.yaml
+testdata/errors/UnknownScalar.schema.graphql:3: unknown scalar UnknownScalar: please add it to "bindings" in genqlient.yaml

generate/testdata/snapshots/TestGenerateErrors-UnknownScalar.graphql

@@ -1 +1 @@
-testdata/errors/UnknownScalar.schema.graphql:3: unknown scalar UnknownScalar: please add it to genqlient.yaml
+testdata/errors/UnknownScalar.schema.graphql:3: unknown scalar UnknownScalar: please add it to "bindings" in genqlient.yaml

generate/types.go

@@ -46,8 +46,8 @@ var (
 )
 
 type (
-	// goOpaqueType represents a user-defined or builtin type, used to
-	// represent a GraphQL scalar.
+	// goOpaqueType represents a user-defined or builtin type, often used to
+	// represent a GraphQL scalar.  (See Config.Bindings for more context.)
 	goOpaqueType struct{ GoRef string }
 	// goSliceType represents the Go type []Elem, used to represent GraphQL
 	// list types.

internal/testutil/types.go

@@ -1,3 +1,12 @@
 package testutil
 
 type ID string
+
+type Pokemon struct {
+	Species string `json:"species"`
+	Level   int    `json:"level"`
+}
+
+func (p Pokemon) Battle(q Pokemon) bool {
+	return p.Level > q.Level
+}