It has been an old request, it is now available!
I just published a nuget package that includes the latest version of XrmToolBox executable and the connection libraries. When you want to develop your own plugin, you can use the Visual Studio Template available in the XrmToolBox donwload page. But there is still missing references.
To resolve this problem, I published a nuget package that will import these missing reference to your Visual Studio project.
To install the package in Visual Studio, right click on the references folder and “Manage nuget packages”.
Then search for “XrmToolBox”, you will find the package. Install it!