Try searching for "kvm over ip" -- you'll find a number of devices not much more expensive than that adapter, which are KVMs with their own network interfaces. Which means you don't even need to be physically plugged in to use them, though I believe you *could* be if security concerns mean you don't want to connect the KVM to a public network.
thank you