Writing native database drivers - and a new MySQL driver port
Writing native database drivers - and a new MySQL driver port
If there is an open source driver already available in C or D, the process is straight forward. Since vibe.d uses a blocking I/O model and almost any database driver is written using blocking sockets, all that is needed is to replace any socket by a TcpConnection. The following table shows the most common socket functions next their vibe.d equivalent.
vibe.d | D sockets | C sockets |
---|---|---|
TcpConnection |
Socket/TcpSocket |
SOCKET/int |
TcpConnection.tcpNoDelay |
Socket.setOption(SocketOption.TCP_NODELAY) |
setsockopt(TCP_NODELAY) |
connectTcp(host, port) |
Socket.connect(addr) |
connect(sock, addr) |
TcpConnection.write(buf) |
Socket.send(buf) |
send(sock, buf, len, flags) |
TcpConnection.read(buf) |
Socket.receive(buf) |
recv(sock, buf, len, flags) |
TcpConnection.close() |
Socket.shutdown(which), Socket.close() |
shutdown(which), close() |
Note that unlike the C and D versions, TcpConnection.read()
and write()
will always read/write the whole buffer that is given. If this is not possible for some reason, they will throw an exception. Because of this, it is often possible to simplify the code a bit and remove the otherwise needed loops.
As an example, the following list shows the changes that were necessary to port the MySQL driver originally written in D by Steve Teale to the vibe framework. MySQL uses a packet based serial protocol over a TCP connection to communicate between the server and its clients.
The first part was to replace the import and the socket declarations:
import std.socket; // ... class Connection { // ... Socket _socket; ubyte[] _packet; // ...
was changed to:
import vibe.core.tcp; // ... class Connection { // ... TcpConnection _socket; ubyte[] _packet; // ...
Then the connection sequence had to be adjusted:
_socket = new TcpSocket(); Address a = new InternetAddress(InternetAddress.PORT_ANY); _socket.bind(a); a = new InternetAddress(_host, _port); _socket.setOption(SocketOptionLevel.SOCKET, SocketOption.RCVBUF, (1 << 24)-1); _socket.connect(a);
this was replaced by the following line. The
RECVBUF
option is not available in vibe, but is also not needed because its effects are mostly hidden inside the system._socket = connectTcp(_host, _port);
Next, there were a lot of single calls to
_socket.receive()
and_socket.write()
. These were replaced by calls toread()/write()
with the same parameters. Note that the code actually was made more robust here, because the return value of the original calls was seldomly checked and this way data could get lost (this probably did not actually happen because the read/write buffers of the socket were large enough).However, there was an exception in the code, where this behavior was actually wanted. The transformed code simulates the behavior using
_socket.leastSize
- this waits until data is available an then returns the size of this data._packet.length = 512; _socket.receive(_packet);
was changed to
_packet.length = _socket.leastSize; _socket.read(_packet);
The main
getPacket()
function was performing a loop to read a full protocol packet that was possibly fragmented:_packet.length = pl; ubyte[] buf; buf.length = (pl > _rbs)? _rbs: pl; uint got = 0; size_t n = _socket.receive(buf); for (;;) { _packet[got..got+n] = buf[0..n]; got += n; if (got >= pl) break; if (pl-got < _rbs) buf.length = pl-got; n =_socket.receive(buf); }
This code block could also be replaced by a single line:
_packet.length = pl; _socket.read(_packet);
Finally, the call to
_socket.shutdown()
could be removed becauseTcpConnection.close()
does this implicitly._socket.shutdown(SocketShutdown.BOTH); _socket.close();
became:
_socket.close();
And that's all. The complete converted code is available as a github fork. You can include it in your project by putting it as a dependency in your package.json (See the docs):
{
...
"dependencies": [
"mysql-native": ">=0.0.1"
]
}
It seems to be able to perform a successful login by now, but I have not really tested it any further, yet. It will also need some refactoring and a connection pool to make it more efficient and better fitting into the framework. Thanks to Steve Teale for making the original code available!
Comments for the post are currently disabled.
2 comments
Sorry for the late answer. But yes the sockets need to be non-blocking and they need to be able to wake the event loop of the active event driver (libevent currently).
With standard sockets, the application performance would come to a crawl as soon as many connections are active.
When using vibe.d, is there a problem with using a MySQL lib that's not ported to vibe? Do the standard D and C sockets conflict with vibe.d's sockets?