If you're saying "the GPU hardware may execute hardware instructions by breaking them down internally to even simpler operations, in the same way that modern x86 CPUs execute x86 instructions by breaking them down to simpler operations" then I guess I agree. That doesn't mean there is an internal instruction set you could potentially program in though.
I don't work for NVidia ;)
That said, I think you'll find that both NVidia and AMD GPUs today are more capable than what you are describing. I know that SI is, and I imagine the corresponding NVidia parts aren't too different.

