Ok now it uses MPS. I had to carefully make sure that I have Go for arm64 set up properly. "brew bundle install go" mysteriously didn't work but "brew install go" did. I haven't looked into why.

Now my local build uses the GPU similar to the downloaded version!