This is the local socket for windows, analogous to AF_UNIX.
`Efl_Net_Socket_Windows` is the base class doing `ReadFile()` and
`WriteFile()` using overlapped I/O, as well as the close procedure
(`FlushFileBuffers()`, `DisconnectNamedPipe()` and
`CloseHandle()`). These are done on top of an existing HANDLE that is
set by `Efl_Net_Dialer_Windows` (from `CreateFile()`) or
`Efl_Net_Server_Windows` (from `CreateNamedPipe()`).
The overlapped I/O will return immediately, either with operation
completed or `ERROR_IO_PENDING`, which means the kernel will execute
that asynchronously and will later `SetEvent(overlapped.hEvent)` which
is an event we wait on our main loop. That `overlapped` handle must
exist during the call lifetime, thus cannot be bound to `pd`, as we
may call `CancelIo()` but there is no guarantee the memory won't be
touched, in that case we keep the overlapped around, but without an
associated object.
Windows provides no notification "can read without blocking" or
non-blocking calls that returns partial data. The way to go is to use
these overlapped I/O, with an initial `ReadFile()` to an internal
buffer, once that operation finishes, we callback the user to says
there is something to read (`efl_io_reader_can_read_set()`) and wait
until `efl_io_reader_read()` is called to consume the available data,
then `ReadFile()` is called again to read more data to the same
internal buffer.
Likewise, there is no "can write without blocking" or non-blocking
calls that sends only partial data. The way to go is to get user bytes
in `efl_io_writer_write()` and copy them in an internal buffer, then
call `WriteFile()` on that and inform the user nothing else can be
written until that operation completes
(`efl_io_writer_can_write_set()`).
This is cumbersome since we say we "sent" stuff when we actually
didn't, it's still in our internal buffer (`pd->send.bytes`), but
nonetheless the kernel and the other peer may be adding even more
buffers, in this case we need to do a best effort to get it
delivery. A particular case is troublesome: `write() -> close()`, this
may result in `WriteFile()` pending, in this case we wait using
`GetOverlappedResult()`, *this is nasty and may block*, but it's the
only way I see to cope with such common use case.
Other operations, like ongoing `ReadFile()` or `ConnectNamedPipe()`
will be canceled using `CancelIo()`.
Q: Why no I/O Completion Port (IOCP) was used? Why no
CreateThreadpoolIo()? These perform much better!
A: These will call back from secondary threads, but in EFL we must
report back to the user in order to process incoming data or get
more data to send. That is, we serialize everything to the main
thread, making it impossible to use the benefits of IOCP and
similar such as CreateThreadpoolIo(). Since we'd need to wakeup the
main thread anyways, using `OVERLAPPED.hEvent` with
`ecore_main_win32_handler_add()` does the job as we expect.
Thanks to Vincent Torri (vtorri) for his help getting this code done
with an example on how to do the NamedPipe handling on Windows.
remove one more TODO: since Efl.Net.Ip.Address was introduced we can
now expose Efl.Net.Socket.Udp.init as a protected method that will
configure the internal address we use for the remote peer. This allow
subclasses to override or call such methods.
This is a string parser, serializer and asynchronous resolver.
It's purpose is to convert to and from the strings we use in our
dialers and servers, such as "127.0.0.1:1234" or "[::1]:1234",
properties allow to check the family, port, address bytes (slice) and
even get a struct sockaddr pointer to use with bind()/connect() in
outside code.
It will also offer some utilities present in netinet/in.h in an easy
to use way, after all IN6_IS_ADDR_LOOPBACK() works one way, while
there is no IN_LOOPBACK and comparing with INADDR_LOOPBACK will lead
to errors since it's in network order.
Last but not least, it will do asynchronous resolve of host and port
names using an internal thread and getaddrinfo(). The results are
delivered using a Future with an array of objects.
This is a major work and unfortunately couldn't be split into smaller
pieces as old code was highly coupled.
Ecore_Con_Server is now a wrapper around Efl_Net_Dialer_Simple
(ecore_con_server_connect()) and Efl_Net_Server_Simple
(ecore_con_server_add()), doing all that the original version did with
some fixes so ecore_con_ssl_server_upgrade() and
ecore_con_ssl_client_upgrade() are more usable -- see the examples and
-t/--type=tcp+ssl.
I tried to be bug-compatible, with code annotations where things
doesn't make sense. This was based on ecore_con_suite tests and some
manual experimenting with the examples, these can be helpful if you
find regressions (report/assign to me).
Rewrite Ecore_Con_Url as a non-Eo (since it's just legacy) that is
built on top of Efl_Net_Dialer_Http.
Since there are some legacy behavior we do not want to expose in the
new classes, hack around and manipulate the curl_easy_setopt()
directly in those cases.
This includes the cookies: there is no reason why we should expose
independent files for read (COOKIEFILE) and write (COOKIEJAR), real
world applications can manipulate the files directly, like copying
from a template to a RDWR before using, etc.
The low level I/O primitives are powerful but adds some complexity to
use, for bi-directional streaming communication one ends creating two
Efl.Io.Queue and two Efl.Io.Copier to pipe data to socket when it can
operate.
Then encapsulate the socket using the new Efl.Io.Buffered_Stream, this
will allow the socket, be a dialer or a server client, to be operated
as a single handle that internally carries about the buffering for
you.
As one can see in the examples, compared to their "manual"
alternatives they are very easy to use, ressembling
Ecore_Con_Server/Ecore_Con_Client, but also offers line-based
delimiters and the possibility to let the socket to handle queueing
for you in case you received partial messages (just do not
read/clear/discard the received data).
These are objects to allow control of networking devices
(efl_net_control) as well as an application to request for
connectivity (efl_net_session).
They are loosely based on ConnMan.org, which we already use in
Enlightenment Window Manager via DBus access with Eldbus. However they
do not map 1:1 as the goal was to expose a viable subset of controls
but in a simple and general way, thus nome strings were converted to
enums, some arrays of strings were converted to bitwise flags, some
names were made more general, such as "service" was turned into
"access point" so it doesn't generate confusion with other "network
services" (ie: http server), or "favorite" that was renamed to
"remembered". Some behavior are slightly different (yet able to be
implemented on top), such as "Service.MoveBefore" and "MoveAfter" were
converted to a numeric "priority", calculated from service's list
index, changing the priority will reoder the list and thus generate
the MoveBefore and MoveAfter DBus commands.
ConnMan was chosen not only because we already use it, but because its
DBus API is sane and simple, with the server doing almost all that we
need. This is visible in the efl_net_session, which is completely done
in the server and do not require any extra work on our side -- aside
from talking DBus and converting to Eo, which is a major work :-D
NOTE: ConnMan doesn't use FreeDesktop.Org DBus interfaces such as
Properties and ObjectManager, thus we cannot use
eldbus_model_object.
There are two examples added:
- efl_net_session_example: monitors the connection available for an
application and try to connect. You need a connman compiled with
session_policy_local and a configuration file explained in
https://github.com/aldebaran/connman/blob/master/doc/session-policy-format.txt
to get a connection if nothing is connected. Otherwise it will just
monitor the connectivity state.
- efl_net_control_example: monitors, plays the agent and configure
the network details. It can enable/disable technologies, connect to
access points (services) and configure them. It's quite extensive
as allows testing all of ConnMan's DBus API except P2P (Peers).
in the previous commit we're manually upgrading an existing TCP socket
to SSL. It is desired since some protocols need to negotiate, like
STARTTLS and the likes
Now we offer 2 classes that does autostart SSL once the socket is
ready.
This introduces AF_UNIX server and dialer, these are not available on
Windows as in that platform we'll create a custom class for native
'local' communication.
In the future we can add a wrapper class Efl.Net.Local that will use
the class for each platform, but won't expose its details.
For instance, if we ever expose 'credentials' (which I didn't because
they are not portable), then it doesn't make sense to try to match
that on Windows. The 'Efl.Net.Local' would just stick to the basics:
Reader, Writer and Closer APIs.
This is the initial UDP server that works similarly to the TCP one,
however under the hood it's widely different since the socket is
reused for all "clients", thus needs a new Efl.Net.Server.Udp.Client
(Efl.Net.Socket) as Efl.Net.Socket.Udp exposes the fd and options such
as 'cork', which would interfere in other clients.
The main socket will read the packets and find an existing client to
feed it. If no client exists, then it will create one if not overr
limit. Since there is no kernel-queuing as done by listen()/accept(),
the 'no reject' case will just accept the client anyway.
Next commits will improve UDP server handling with some advanced
features:
- join multicast groups
- bind to a specific interface (SO_BINDTODEVICE)
- block packets going out of local network (SO_DONTROUTE)
- specify priorities (SO_PRIORITY)
Like existing ecore_con code, this does not use SOCKSv5 UDP
proxy. It's kinda cumbersome to add since requires a keep alive TCP
connection to the server, a second UDP channel and framing around the
original UDP frame.
Added UDP_CORK (if present) to match TCP_UDP present in TCP sockets,
this allows one to execute multiple write() calls that will result in
a single datagram, generated when CORK becomes FALSE again.
The efl_io_copier_example.c now accepts this as output. There is no
input UDP as there is no way to notify the server of a connection
(since such thing doesn't exit), usually servers react after a
datagram is received, replying to the source.
SOCKS is implemented in its own thread using synchronous/blocking
primitives, which simplifies the code a lot -- as well as simulate the
usage of Ecore_Thread as our users will likely do.
Since SOCKSv4a and SOCKSv5 allow name resolution, the whole
getaddrinfo() is done in the same thread, when needed, instead of a
separate thread to do that, which should also save some resources.
Instead of the legacy ECORE_CON_SOCKS_V4 and ECORE_CON_SOCKS_V5, now
we use socks_proxy, all_proxy and no_proxy. This matches our other
dialers http/websocket (which will use http_proxy, all_proxy and
no_proxy). If desired it's easy to add back support for those
variables, but I think we should just deprecate them. (The legacy code
will keep unchanged, thus direct users of ecore_con_server will still
use those -- just the previous users of ecore_con_server will be
converted to use the new API).
both resolve (getaddrinfo()) and connect() are now done in
Ecore_Thread, avoid to block the main loop.
My plan is to always use the threaded connect() using a blocking
socket, only set it to non-blocking after the socket is returned to
the main thread and before it's accessible to the user. It will make
the connect behavior more uniform.
Some errors were moved from HTTP to Dialer as they are more generic.
The Efl.Net.Dialer.Websocket is just like other Efl.Net.Dialers: you
can dial, you can close, monitor connected/address resolved and so
on. And you can use WebSocket primitives and events such as
text_send(), binary_send(), ping() and close_request() (since
WebSockets use a close process where you should state a close
reason). See efl_net_dialer_websocket_example.c
Even if WebSocket is a message-based protocol (like "packets" from
UDP), you can use efl_net_dialer_websocket_streaming_mode_set() to
tell it to handle text or binary messages as a stream. Then all the
Efl.Io.Reader and Efl.Io.Writer APIs work as expected, see
efl_io_copier_example.c updates.
This class implements the Efl.Net.Dialer interface using libcurl to
perform HTTP requests. That means it's an Efl.Net.Dialer,
Efl.Net.Socket, Efl.Io.Reader, Efl.Io.Writer and Efl.Io.Closer, thus
being usable with Efl.Io.Copier as demonstrated in the
efl_io_copier_example.c
Efl.Net.Server defines how to accept new connections, doing the
bind(), listen() and accept() for protocols such as TCP.
Efl.Net.Dialer defines to to reach a server.
Both are based on Efl.Net.Socket as communication interface that is
based on Efl.Io.Reader, Efl.Io.Writer and Efl.Io.Closer, thus being
usable with code such as Efl.Io.Copier.
The Server will emit an event "client,add" with the established
Socket, which is a child and can be closed by both the server or the
user.
The Dialer extends the Socket and allows for creating one given an
address, that will be resolved and connected.
TCP is the initial implementation so we an validate the
interfaces. UDP, Unix-Local and SSL will come later as derivate
classes.
The examples are documented and should cover the basic principles:
- efl_io_copier_example can accept "tcp://IP:PORT" and will work as a
"netcat", can send data from socket, file or stdin to a socket,
file, stdout or stderr.
- efl_net_server_example listens for connections and can either reply
"Hello World!" and take some data or work as an echo-server,
looping back all received data to the user.
More complex interactions that require a "chat" between client and
server will be covered with new classes later, such as a queue that
empties itself once data is read.
Summary:
Changed ecore_con_connector.eo to efl_network_connector.eo as part of
migrating to efl_network.
Signed-off-by: Srivardhan Hebbar <sri.hebbar@samsung.com>
Reviewers: cedric, jpeg
Differential Revision: https://phab.enlightenment.org/D3427
Signed-off-by: Cedric BAIL <cedric@osg.samsung.com>
Summary: This is just the beginning. I tried for one class to check.
Tell me if this is fine, I'll change in other classes also. The goal
is to simplify and make our API clearer to understand to new comers.
Reviewers: cedric
Subscribers: cedric
Differential Revision: https://phab.enlightenment.org/D2468
Signed-off-by: Cedric BAIL <cedric@osg.samsung.com>
Gustavo Sverzut Barbieri
Daniel Kolesa
Tom Hacohen
Srivardhan Hebbar
Mike Blumenkrantz