Hybrid networking
Three sites, one VPN hub
Connect on-premise, AWS and Azure as if they were on the same LAN, with open-source SoftEther.
Kevin Keller · kevinkeller.org
Why it matters
Most enterprises end up with networks in three places
- On-premise, one cloud, then another cloud.
- The networks are isolated by default.
- Cloud vendor VPN appliances cost money and lock you in.
- SoftEther is open source and supports every major VPN protocol.
The architecture
Every site dials into a central hub with a public IP
| Site | Network | Connects via |
|---|---|---|
| On-premise | 192.168.1.0/24 | VPN client on a gateway |
| AWS | 10.1.0.0/16 | SoftEther client on EC2 |
| Azure | 10.2.0.0/16 | SoftEther client on a VM |
| Hub | 192.168.30.0/24 | Ubuntu server, public IP |
The idea
The hub forwards Ethernet frames, not subnets
SoftEther bridges at Layer 2, so all sites share one virtual switch. The hub doesn't need to know about individual subnets.
Building the hub
Four pieces turn one server into a router for all sites
- SoftEther server with a virtual hub, a TAP bridge and one user per site.
- IP forwarding enabled in the kernel.
- iptables rules, including TAP-to-TAP forwarding between clients.
- dnsmasq for DHCP and DNS on the VPN interface.
The key trick
DHCP pushes routes to every site automatically
- dnsmasq advertises all three site subnets via the hub at
192.168.30.1. - Option 121 for Linux and Mac clients, option 249 for Windows.
- Optional
/etc/hostsentries give cross-site internal hostnames.
Connecting the clouds
Each cloud gateway needs one setting people forget
| Site | Gateway | Must-have |
|---|---|---|
| AWS | Small EC2 (t3.micro) | Source/dest check disabled, VPC route table entry |
| Azure | Small VM (B1s) | IP forwarding on the NIC, user-defined route |
| On-premise | Any Linux gateway | Router sends cloud subnets to it |
The routing flow
An on-premise machine reaching AWS takes a predictable path
- The packet hits the on-premise gateway.
- It enters the SoftEther tunnel to the hub.
- The hub forwards it on
tap_softto the AWS client. - The AWS gateway delivers it inside the VPC.
- The response follows the same path back.
Security
A hub that connects everything needs tight controls
- Per-site users: revoke one site without touching others
- Management port 5555: trusted IPs only
- Prefer certificates over passwords and pre-shared keys
- Protect the config file:
chmod 600 - Audit sessions regularly with
SessionList
The takeaway
A single SoftEther hub gives you a fully routed mesh across on-premise, AWS and Azure.
Kevin Keller · kevinkeller.org