Ensure that this project has Microsoft.Bcl.Build installed and packages.config is located next to the project file

I received this error while building a UWP app.
Could not locate xxxx\packages.config. Ensure that this project has Microsoft.Bcl.Build installed and packages.config is located next to the project file.
I did check and see the package.config in the right directory and I did have Microsoft.BCL.Build. The only way to get this to work is to close visual studio delete the obj folder in the solutions directory. Once the obj folder is deleted and I reloaded the project in visual studio everything worked again.