Last updated
Last updated
golang challenge: write a function
walk(x interface{}, fn func(string))which takes a structxand callsfnfor all strings fields found inside. difficulty level: recursively.
To do this we will need to use reflection.
Reflection in computing is the ability of a program to examine its own structure, particularly through types; it's a form of metaprogramming. It's also a great source of confusion.
From
interface{}?We have enjoyed the type-safety that Go has offered us in terms of functions that work with known types, such as string, int and our own types like BankAccount.
This means that we get some documentation for free and the compiler will complain if you try and pass the wrong type to a function.
You may come across scenarios though where you want to write a function where you don't know the type at compile time.
Go lets us get around this with the type interface{} which you can think of as just any type (in fact, in Go any is an for interface{}).
So walk(x interface{}, fn func(string)) will accept any value for x.
interface{} for everything and have really flexible functions?As a user of a function that takes interface{} you lose type safety. What if you meant to pass Herd.species of type string into a function but instead did Herd.count which is an int? The compiler won't be able to inform you of your mistake. You also have no idea what you're allowed to pass to a function. Knowing that a function takes a UserService for instance is very useful.
As a writer of such a function, you have to be able to inspect anything that has been passed to you and try and figure out what the type is and what you can do with it. This is done using reflection. This can be quite clumsy and difficult to read and is generally less performant (as you have to do checks at runtime).
In short only use reflection if you really need to.
If you want polymorphic functions, consider if you could design it around an interface (not interface{}, confusingly) so that users can use your function with multiple types if they implement whatever methods you need for your function to work.
Our function will need to be able to work with lots of different things. As always we'll take an iterative approach, writing tests for each new thing we want to support and refactoring along the way until we're done.
We'll want to call our function with a struct that has a string field in it (x). Then we can spy on the function (fn) passed in to see if it is called.
We want to store a slice of strings (got) which stores which strings were passed into fn by walk. Often in previous chapters, we have made dedicated types for this to spy on function/method invocations but in this case, we can just pass in an anonymous function for fn that closes over got.
We use an anonymous struct with a Name field of type string to go for the simplest "happy" path.
Finally, call walk with x and the spy and for now just check the length of got, we'll be more specific with our assertions once we've got something very basic working.
We need to define walk
Try and run the test again
We can call the spy with any string to make this pass.
The test should now be passing. The next thing we'll need to do is make a more specific assertion on what our fn is being called with.
Add the following to the existing test to check the string passed to fn is correct
This code is very unsafe and very naive, but remember: our goal when we are in "red" (the tests failing) is to write the smallest amount of code possible. We then write more tests to address our concerns.
We need to use reflection to have a look at x and try and look at its properties.
We then make some very optimistic assumptions about the value passed in:
We look at the first and only field. However, there may be no fields at all, which would cause a panic.
We then call String(), which returns the underlying value as a string. However, this would be wrong if the field was something other than a string.
Our code is passing for the simple case but we know our code has a lot of shortcomings.
We're going to be writing a number of tests where we pass in different values and checking the array of strings that fn was called with.
We should refactor our test into a table based test to make this easier to continue testing new scenarios.
Now we can easily add a scenario to see what happens if we have more than one string field.
Add the following scenario to the cases.
val has a method NumField which returns the number of fields in the value. This lets us iterate over the fields and call fn which passes our test.
It doesn't look like there's any obvious refactors here that would improve the code so let's press on.
The next shortcoming in walk is that it assumes every field is a string. Let's write a test for this scenario.
Add the following case
We need to check that the type of the field is a string.
Again it looks like the code is reasonable enough for now.
The next scenario is what if it isn't a "flat" struct? In other words, what happens if we have a struct with some nested fields?
We have been using the anonymous struct syntax to declare types ad-hocly for our tests so we could continue to do that like so
Let's just refactor this by making a known type for this scenario and reference it in the test. There is a little indirection in that some of the code for our test is outside the test but readers should be able to infer the structure of the struct by looking at the initialisation.
Add the following type declarations somewhere in your test file
Now we can add this to our cases which reads a lot clearer than before
The problem is we're only iterating on the fields on the first level of the type's hierarchy.
The solution is quite simple, we again inspect its Kind and if it happens to be a struct we just call walk again on that inner struct.
When you're doing a comparison on the same value more than once generally refactoring into a switch will improve readability and make your code easier to extend.
What if the value of the struct passed in is a pointer?
Add this case
You can't use NumField on a pointer Value, we need to extract the underlying value before we can do that by using Elem().
Let's encapsulate the responsibility of extracting the reflect.Value from a given interface{} into a function.
This actually adds more code but I feel the abstraction level is right.
Get the reflect.Value of x so I can inspect it, I don't care how.
Iterate over the fields, doing whatever needs to be done depending on its type.
Next, we need to cover slices.
This is similar to the pointer scenario before, we are trying to call NumField on our reflect.Value but it doesn't have one as it's not a struct.
This works but it's yucky. No worries, we have working code backed by tests so we are free to tinker all we like.
If you think a little abstractly, we want to call walk on either
Each field in a struct
Each thing in a slice
Our code at the moment does this but doesn't reflect it very well. We just have a check at the start to see if it's a slice (with a return to stop the rest of the code executing) and if it's not we just assume it's a struct.
Let's rework the code so instead we check the type first and then do our work.
Looking much better! If it's a struct or a slice we iterate over its values calling walk on each one. Otherwise, if it's a reflect.String we can call fn.
Still, to me it feels like it could be better. There's repetition of the operation of iterating over fields/values and then calling walk but conceptually they're the same.
If the value is a reflect.String then we just call fn like normal.
Otherwise, our switch will extract out two things depending on the type
How many fields there are
How to extract the Value (Field or Index)
Once we've determined those things we can iterate through numberOfValues calling walk with the result of the getField function.
Now we've done this, handling arrays should be trivial.
Add to the cases
Arrays can be handled the same way as slices, so just add it to the case with a comma
The next type we want to handle is map.
Again if you think a little abstractly you can see that map is very similar to struct, it's just the keys are unknown at compile time.
However, by design you cannot get values out of a map by index. It's only done by key, so that breaks our abstraction, darn.
How do you feel right now? It felt like maybe a nice abstraction at the time but now the code feels a little wonky.
This is OK! Refactoring is a journey and sometimes we will make mistakes. A major point of TDD is it gives us the freedom to try these things out.
By taking small steps backed by tests this is in no way an irreversible situation. Let's just put it back to how it was before the refactor.
We've introduced walkValue which DRYs up the calls to walk inside our switch so that they only have to extract out the reflect.Values from val.
Remember that maps in Go do not guarantee order. So your tests will sometimes fail because we assert that the calls to fn are done in a particular order.
To fix this, we'll need to move our assertion with the maps to a new test where we do not care about the order.
Here is how assertContains is defined
Since we have extracted maps into a new test, we haven't seen the failure message. Intentionally break the with maps test here so that you can check the error message, then fix it again so all tests are passing.
The next type we want to handle is chan.
We can iterate through all values sent through channel until it was closed with Recv()
The next type we want to handle is func.
Non zero-argument functions do not seem to make a lot of sense in this scenario. But we should allow for arbitrary return values.
Introduced some concepts from the reflect package.
Used recursion to traverse arbitrary data structures.
Did an in retrospect bad refactor but didn't get too upset about it. By working iteratively with tests it's not such a big deal.
Now that you know about reflection, do your best to avoid using it.
The has a function ValueOf which returns us a Value of a given variable. This has ways for us to inspect a value, including its fields which we use on the next line.
We can do that by checking its .
But we can see that when you get inner anonymous structs the syntax gets a little messy. .
This only covered a small aspect of reflection. .