in the previous commit we're manually upgrading an existing TCP socket
to SSL. It is desired since some protocols need to negotiate, like
STARTTLS and the likes
Now we offer 2 classes that does autostart SSL once the socket is
ready.
Clang raised a massive amount of warnings due to the struct sockaddr_un
not being declared before using it. So, include the header that declares
this structure first.
for short downloads the CURL handle will be done before the client had
time to read it, like done by efl_io_copier's job. We need to wait it
drain before we emit eos/closed.
This introduces AF_UNIX server and dialer, these are not available on
Windows as in that platform we'll create a custom class for native
'local' communication.
In the future we can add a wrapper class Efl.Net.Local that will use
the class for each platform, but won't expose its details.
For instance, if we ever expose 'credentials' (which I didn't because
they are not portable), then it doesn't make sense to try to match
that on Windows. The 'Efl.Net.Local' would just stick to the basics:
Reader, Writer and Closer APIs.
This was a huge work, but now UDP is usable as seen in the examples.
Instead of relying on 'connect()', just do 'sendto()' and 'recvfrom()'
as they are universal. Multicast address can only be connected in
IPv4, IPv6 wasn't working and I'm not sure the IPv4 is portable to
other platforms.
Dialer will auto-join multicast groups is the dialed address is
one. Multicast properties such as time to live (hops) and loopback can
be configured. When joining multicast groups, the local
address/interface can be configured by 'IP@IFACE' format, with
'@IFACE' being optional.
Dialers will now auto-bind, so it can receive data as dialers are
expected to be bi-directional. One can manually specify the binding
address if there is such need.
Since datagrams must be read in their full size, otherwise the
remaining bits are dropped, expose next_datagram_size_query() in both
Efl.Net.Socket.Udp and Efl.Net.Server.Udp.Client.
To finalize UDP for real we need to introduce an 'Efl_Net_Ip_Address'
structure to serve as both IPv4 and IPv6 and expose 'sendto()' and
'recvfrom()'. These will come later as this commit is already too big.
Thanks to vtorri for poiting out about close() is not the correct
socket function, we should use closesocket() instead.
Also defined SOCKET to int on Linux so we can use the same 'type' and
avoid lots of ifdef in our code. On Windows it's unsigned, thus would
cause some warnings about incorrect signed comparison.
Defined INVALID_SOCKET=-1 and SOCKET_ERROR=-1 on non-Windows platforms
so we can keep the same construct 'function() == error' and it should
work on POSIX and windows.
I cannot test these on Windows, but the situation should be improved
with this commit.
if no hints were specified, getaddrinfo() will assume ai_flags as
AI_ADDRCONFIG | AI_V4MAPPED, which only reports useful results based
on what system supports. For instance AI_ADDRCONFIG will only return
IPv4 if IPv4 address exists, likewise IPv6 will only be returned if
IPv6 address is configured, avoiding these to be tried and error for
most local networks where such address could not be used. AI_V4MAPPED
will map IPv4 address over IPv6 if no IPv4 was found.
This is the initial UDP server that works similarly to the TCP one,
however under the hood it's widely different since the socket is
reused for all "clients", thus needs a new Efl.Net.Server.Udp.Client
(Efl.Net.Socket) as Efl.Net.Socket.Udp exposes the fd and options such
as 'cork', which would interfere in other clients.
The main socket will read the packets and find an existing client to
feed it. If no client exists, then it will create one if not overr
limit. Since there is no kernel-queuing as done by listen()/accept(),
the 'no reject' case will just accept the client anyway.
Next commits will improve UDP server handling with some advanced
features:
- join multicast groups
- bind to a specific interface (SO_BINDTODEVICE)
- block packets going out of local network (SO_DONTROUTE)
- specify priorities (SO_PRIORITY)
this allows nicer usage such as 'localhost:http' as the address, which
will resolve to [::1]:80 (if IPv6 is enabled) or 127.0.0.1:80 if only
IPv4 exists.
Sometimes we want to handle both IPv4 and IPv6 in the same socket,
instead of spawning 2 servers, one for each protocol. That is achieved
by means of disabling IPV6_V6ONLY socket option, present in most
recent platforms.
Like other toolkits, let's enable this automatically for users before
connecting to 255.255.255.255 IPv4 (IPADDR_BROADCAST), otherwise most
systems will just fail to connect and send packets.
Like existing ecore_con code, this does not use SOCKSv5 UDP
proxy. It's kinda cumbersome to add since requires a keep alive TCP
connection to the server, a second UDP channel and framing around the
original UDP frame.
Added UDP_CORK (if present) to match TCP_UDP present in TCP sockets,
this allows one to execute multiple write() calls that will result in
a single datagram, generated when CORK becomes FALSE again.
The efl_io_copier_example.c now accepts this as output. There is no
input UDP as there is no way to notify the server of a connection
(since such thing doesn't exit), usually servers react after a
datagram is received, replying to the source.
by default we'll start with cork=0 and on adoption of a FD we'll apply
cached values, thus we'd try to apply cork=0 (default) and it would
error, which is annoying on platforms without such feature.
since users interested in TCP_CORK will enable it first, they will get
the error at that point.
Thanks to Victor Pereira from the SUSE Security team for auditing
this and recommending better options.
This has been discussed several times but knowone ever got to
commiting it.
libproxy allows various means to configure a proxy, will load from
gnome and kde configuration settings, envvars, macos and even windows
registry.
curl still doesn't use it, but we can make that later.
SOCKS is implemented in its own thread using synchronous/blocking
primitives, which simplifies the code a lot -- as well as simulate the
usage of Ecore_Thread as our users will likely do.
Since SOCKSv4a and SOCKSv5 allow name resolution, the whole
getaddrinfo() is done in the same thread, when needed, instead of a
separate thread to do that, which should also save some resources.
Instead of the legacy ECORE_CON_SOCKS_V4 and ECORE_CON_SOCKS_V5, now
we use socks_proxy, all_proxy and no_proxy. This matches our other
dialers http/websocket (which will use http_proxy, all_proxy and
no_proxy). If desired it's easy to add back support for those
variables, but I think we should just deprecate them. (The legacy code
will keep unchanged, thus direct users of ecore_con_server will still
use those -- just the previous users of ecore_con_server will be
converted to use the new API).
Document some proxy behavior like done by CURL, so we'll follow that
standard, with $http_proxy, $socks_proxy, $all_proxy and $no_proxy.
also add some missing @since.
Efl_Future actually work with weak reference. So you do not need to
set things to NULL, but you actually need to register the memory location
of the future with efl_future_use.
As discussed in the mailing list, many people will use worker threads
to execute blocking syscalls and mandating ecore_thread_check() for
voluntary preemption reduces the ecore_thread usefulness a lot.
A clear example is ecore_con usage of connect() and getaddrinfo() in
threads. If the connect timeout expires, the thread will be cancelled,
but it was blocked on syscalls and they will hang around for long
time. If the application exits, ecore will print an error saying it
can SEGV.
Then enable access to pthread_setcancelstate(PTHREAD_CANCEL_ENABLE)
via eina_thread_cancellable_set(EINA_TRUE), to pthread_cancel() via
eina_thread_cancel(), to pthread_cleanup_push()/pthread_cleanup_pop()
via EINA_THREAD_CLEANUP_PUSH()/EINA_THREAD_CLEANUP_POP() and so on.
Ecore threads will enforce non-cancellable threads on its own code,
but the user may decide to enable that and allow cancellation, that's
not an issue since ecore_thread now plays well and use cleanup
functions.
Ecore con connect/resolve make use of that and enable cancellable
state, efl_net_dialer_tcp benefits a lot from that.
A good comparison of the benefit is to run:
./src/examples/ecore/efl_io_copier_example tcp://google.com:1234 :stdout:
before and after. It will timeout after 30s and with this patch the
thread is gone, no ecore error is printed about possible SEGV.
In case when _ecore_con_ssl_client_init_(gnutls/openssl) finished
successful a enum ECORE_CON_SSL_ERROR_NONE value (0) returned. Function
ecore_con_ssl_client_upgrade return Eina_Bool and in case of success
EINA_FALSE was returned.
@fix
both resolve (getaddrinfo()) and connect() are now done in
Ecore_Thread, avoid to block the main loop.
My plan is to always use the threaded connect() using a blocking
socket, only set it to non-blocking after the socket is returned to
the main thread and before it's accessible to the user. It will make
the connect behavior more uniform.
Some errors were moved from HTTP to Dialer as they are more generic.
it seems that on windows read() and write() won't work with sockets,
so use recv() and send().
Note that this code is still untested on windows, at least the errors
must be fetched using WSAGetLastError() instead of errno directly, but
I don't have a Windows machine I can test.
It has been discussed on the ML (thread: "[RFC] rename efl_self") and
IRC, and has been decided we should rename it to this in order to avoid
confusion with the already established meaning of self which is very
similar to what we were using it for, but didn't have complete overlap.
Kudos to Marcel Hollerbach for initiating the discussion and
fighting for it until he convinced a significant mass. :)
This commit breaks API, and depending on compiler potentially ABI.
@feature
The Efl.Net.Dialer.Websocket is just like other Efl.Net.Dialers: you
can dial, you can close, monitor connected/address resolved and so
on. And you can use WebSocket primitives and events such as
text_send(), binary_send(), ping() and close_request() (since
WebSockets use a close process where you should state a close
reason). See efl_net_dialer_websocket_example.c
Even if WebSocket is a message-based protocol (like "packets" from
UDP), you can use efl_net_dialer_websocket_streaming_mode_set() to
tell it to handle text or binary messages as a stream. Then all the
Efl.Io.Reader and Efl.Io.Writer APIs work as expected, see
efl_io_copier_example.c updates.
When CURLOPT_WRITEFUNCTION returns less then the requested amount,
CURL will fail, not call us back with the remaining data.
Then in such cases we must pause CURL and read nothing.
When unpausing we need to kick curl with timeout action so FD handlers
will be re-arranged.
Last but not least, sync our buffer limit with CURL, otherwise it may
always fail if we're smaller than CURL.
CURL doesn't play nice if handles are deleted or modified while it's
dispatching the callbacks, then we must not touch the CURL* easy
handle in those cases, just dissociate the handle from object and
schedule a job to do the deletion later.
Also, since from CURL callbacks we do not have the reference to the
object, if they are deleted from inside the callback, users of 'pd'
will crash. Thus keep an extra reference while the object and its
private data are in use.
The curl_multi_info_read() is used to notify of errors and
end-of-stream, if we do callback directly from there, the user may
efl_del(dialer), which will result in the "pd->easy" being destroyed
with curl_easy_cleanup() then "cm" and "cm->multi" being destroyed.
Thus postpone that action and keep a list of finished objects, calling
their event handlers which can delete the object (or siblings), thus
ref before dispatching and unref afterwards, taking care to monitor
EFL_EVENT_DEL so we do not use stale objects.
Summary:
The port reuse feature and 'SO_REUSEPORT' flag are not supported by a few linux
In case of linux kernel, it supported from v 3.9
(https://kernelnewbies.org/Linux_3.9)
On the lower version of kernel, compile is failed
Reviewers: barbieri, jayji
Reviewed By: jayji
Subscribers: akanad, id213sin, cedric, jpeg
Differential Revision: https://phab.enlightenment.org/D4256
provide curl with CURLOPT_OPENSOCKETFUNCTION and keep the fd in our
private data.
This is required because on _efl_net_dialer_http_efl_io_writer_write()
we may have no fdhandler.
It happened to me while implementing the WebSocket that uses a
bi-directional communication on top of HTTP and the server sent the
whole message, CURL reads:
recvfrom(7, "...", 16384, 0, NULL, NULL) = 86
recvfrom(7, "", 16384, 0, NULL, NULL) = 0
After the empty (second) recvfrom(), CURL will remove the fdhandler:
DBG:ecore_con lib/ecore_con/efl_net_dialer_http.c:482 _efl_net_dialer_http_curlm_socket_manage() dialer=0x4000000040000005 fdhandler=(nil), fd=7, curl_easy=0x5561846ca8d0, flags=0x4
However I should be able to write to this socket, in my case I need to
reply to a PING request with a PONG.
CURL is smart and when you ask for CURLOPT_HTTPGET, it will
automatically configure UPLOAD=false. Likewise, if you ask for
UPLOAD=1 it will configure CURLOPT_PUT...
However, to do things like WebSocket we need to do a GET request where
we need to send data, then UPLOAD=true must be used.
Then use both information in order to setup the request method and
upload, using CURLOPT_CUSTOMREQUEST to force a given HTTP method.
If we delete the curl multi handle, then we should stop any timer that
was scheduled, otherwise it will use a dead or null pointer.
also add some debug to help track down when the multi handle is
deleted.
Efl.Object.event_callback_call no longer calls legacy smart callbacks;
calling only event callbacks registered with the given event description
pointer.
Create the method Efl.Object.event_callback_legacy_call to inherit the old
behavior from Efl.Object.event_callback_call, calling both Efl.Object events
and legacy smart callbacks.
Update all other files accordingly in order to still supply legacy
callbacks while they are necessary.
TCP_CORK is Linux only. TCP_NOPUSH is supposed to
do the same thing than TCP_CORK, but on BSD (including
Mac OS X).
We now check for the existance of TCP_CORK or TCP_NOPUSH,
and use the right option. If none exist, cork_{set,get}
will just fail.
This class implements the Efl.Net.Dialer interface using libcurl to
perform HTTP requests. That means it's an Efl.Net.Dialer,
Efl.Net.Socket, Efl.Io.Reader, Efl.Io.Writer and Efl.Io.Closer, thus
being usable with Efl.Io.Copier as demonstrated in the
efl_io_copier_example.c
Efl.Net.Server defines how to accept new connections, doing the
bind(), listen() and accept() for protocols such as TCP.
Efl.Net.Dialer defines to to reach a server.
Both are based on Efl.Net.Socket as communication interface that is
based on Efl.Io.Reader, Efl.Io.Writer and Efl.Io.Closer, thus being
usable with code such as Efl.Io.Copier.
The Server will emit an event "client,add" with the established
Socket, which is a child and can be closed by both the server or the
user.
The Dialer extends the Socket and allows for creating one given an
address, that will be resolved and connected.
TCP is the initial implementation so we an validate the
interfaces. UDP, Unix-Local and SSL will come later as derivate
classes.
The examples are documented and should cover the basic principles:
- efl_io_copier_example can accept "tcp://IP:PORT" and will work as a
"netcat", can send data from socket, file or stdin to a socket,
file, stdout or stderr.
- efl_net_server_example listens for connections and can either reply
"Hello World!" and take some data or work as an echo-server,
looping back all received data to the user.
More complex interactions that require a "chat" between client and
server will be covered with new classes later, such as a queue that
empties itself once data is read.
Summary:
Copying from string 'buf' of length 4095 to '&socket_unix.sun_path[0]'
may form a non-terminated C string of size 108. So added null termination.
Signed-off-by: Prateek Thakur <prateek.th@samsung.com>
Reviewers: cedric, thiepha
Subscribers: jpeg
Differential Revision: https://phab.enlightenment.org/D4247
Signed-off-by: Cedric BAIL <cedric@osg.samsung.com>
so ecore-con already cleared the slaves list on shutdown but an
ecore_thread may stillbe active and thus removing from a NULL slaves
list wont work, so skip if slaves is NULL.
@fix
This removes some useless code in various places, where the
switch from eo_do() to standard function call was not properly
refactored.
This changes:
type ret = 0;
ret = my_eo_function();
return ret;
To:
return my_eo_function();
so drop trying to appease the openbsd packages and stick to "upstream
so major versions" and let users fix their systems with symlinks. also
report what we are looking for so they have a chance to symlink to
make efl happy.
at some point we should make a single simple runtime lib linker
subsystem in efl so all these errors are reported in the same way,
input libray names are listed in a simple consistent way etc. etc.
for now we have 3 locations in efl that do this and they are roughly
similar. we can unify it later.
This lets me narrow down the remaining cases of pointers across the EFL.
The void pointers will later need to be reevaluated on per-case basis and
replaced appropriately where possible/feasible.
This reverts commit 546ff7bbba.
It seems that eo_del() is useful and removing it was creating bugs.
The issue is that the way we defined parents in eo, both the parent and
the programmer share a reference to the object. When we eo_unref() that
reference as the programmer, eo has no way to know it's this specific
reference we are freeing, and not a general one, so in some
circumstances, for example:
eo_ref(child);
eo_unref(child); // trying to delete here
eo_unref(container); // container is deleted here
eo_unref(child); // child already has 0 refs before this point.
We would have an issue with references and objects being freed too soon
and in general, issue with the references.
Having eo_del() solves that, because this one explicitly unparents if
there is a parent, meaning the reference ownership is explicitly taken
by the programmer.
eo_del() is essentially a convenience function around "check if has
parent, and if so unparent, otherwise, unref". Which should be used when
you want to delete an object although it has a parent, and is equivalent
to eo_unref() when it doesn't have one.
Summary:
this removes the cares/ares based resolver and the compiled-in dns.c
resolver, modified the getaddrinfo based resolver to use threads not
forking (almost halving its size) and now makes that the only resolver
we have. getaddrinfo handles ipv6 and ipv4 (according to docs). this
simplifies code paths, drops code size of the efl tree by about 11k
lines of code, makes it easier to test and more robust to future
changes with ip resolving as it now just relies on libc. we won't have
coverity complaints on dns.c imported code anymore to fix and don't
have tokeep up with bugfixes/security from the upstream imported code.
this means we use a single resolver on all platforms (windows, mac,
linux) as opposed to before where cares was used for windows, and
dns.c on linux/mac. oh and the forking original was broken since our
move to eo too. so it couldnt even compile if enabled, letalone work.
so fix bug with missing /etc/resolv.conf that dns.c couldn't cope
with, fix testability, fix maintainability and reduce efl codebase size.
this fixes T3668
@fix
@improve
Subscribers: cedric, seoz, jpeg
Maniphest Tasks: T3668
Differential Revision: https://phab.enlightenment.org/D3971
Complex types (i.e. list, array, hash, accessor etc.) now do not require
pointers with them anymore (the pointer is implied) and the same goes for
class handles. Eolian now explicitly disallows creating pointers to these
as well. This is the first part of the work to remove pointers from Eolian
completely, with the goal of simplifying the DSL (higher level) and therefore
making it easier for bindings (as well as easier API usage).
@feature
We used to have eo_del() as the mirrored action to eo_add(). No longer,
now you just always eo_unref() to delete an object. This change makes it
so the reference of the parent is shared with the reference the
programmer has. So eo_parent_set(obj, NULL) can free an object, and so
does eo_unref() (even if there is a parent).
This means Eo no longer complains if you have a parent during deletion.
Mostly unused vars following the removal of eo_do_ret().
However, there are some cases where the migration script got some things
wrong, and I had to manually fix them.
I just ran my script (email to follow) to migrate all of the EFL
automatically. This commit is *only* the automatic conversion, so it can
be easily reverted and re-run.
Summary:
I do not have a windows setup. So tested by building with option --with-windows-version and it built successfully. Let me know if there are more
issues.
Signed-off-by: Srivardhan Hebbar <sri.hebbar@samsung.com>
Reviewers: jpeg, vtorri, cedric
Reviewed By: cedric
Maniphest Tasks: T3192
Differential Revision: https://phab.enlightenment.org/D3708
Signed-off-by: Cedric BAIL <cedric@osg.samsung.com>
This wasn't done correctly in the previous commit. First of all, the
order of cleanup was wrong, the cleanup area should only be called if
failures occurred after the fd allocation, not before. Also, fd should
be reinitialised to -1 once we close the socket.
Summary:
Socket fd must be closed to avoid file discripter leak.
Programs can usually only open a limited number of file descriptors,
so if this happens a lot, it may turn into a problem.
@fix
Reviewers: raster, Hermet, wonsik, spacegrapher, cedric, jpeg, tasn
Reviewed By: tasn
Subscribers: cedric, alok25, yashu21985, singh.amitesh
Differential Revision: https://phab.enlightenment.org/D3660
IPV6_ADD_MEMBERSHIP does not exist on OS X, and seems to be obsolete,
according to my glibc's bits/in.h.
IPV6_JOIN_GROUP, however, exists on both.
@fix
Summary:
Changed ecore_con_connector.eo to efl_network_connector.eo as part of
migrating to efl_network.
Signed-off-by: Srivardhan Hebbar <sri.hebbar@samsung.com>
Reviewers: cedric, jpeg
Differential Revision: https://phab.enlightenment.org/D3427
Signed-off-by: Cedric BAIL <cedric@osg.samsung.com>
To configure efl sources with bindings to use in nodejs add ––with-js=nodejs in configure flags to generate node files
$ configure --with-js=nodejs
and compile normally with:
$ make
$ make install
To use, you have to require efl:
efl = require('efl')
The bindings is divided in two parts: generated and manually
written. The generation uses the Eolian library for parsing Eo files
and generate C++ code that is compiled against V8 interpreter library
to create a efl.node file that can be required in a node.js instance.
@feature
This looks like an obvious case of missing break. If it wasn't a missing
break, there should have been at least a comment. Looking at the code it
looks like a break is needed. Also, I suspect this code path is never
really tested, and that's why we never hit it.
Tests are not failing either way.
CID1039379
The client structure holds a file descriptor, which is not initialized (which means 0) in case of UDP as there is no client-specific socket.
However, we check for the file descriptor being positive before closing it in the client destructor, which means that we actually end up closing the 0 file descriptor.
That means that things were going crazy with real strange things happening afterwards…
Gustavo Sverzut Barbieri
Jean Guyomarc'h
Stefan Schmidt
Daniel Kolesa
Simon Lees
Jean-Philippe Andre
Cedric BAIL
Bruno Dilly
Ji-Youn Park
Mykyta Biliavskyi
Andrii Kroitor
Tom Hacohen
Jiwon Kim
Vitor Sousa
Jean Guyomarc'h
Prateek Thakur
Carsten Haitzler
Al Poole
Mike Blumenkrantz
Daniel Zaoui
Srivardhan Hebbar
Awadhesh Singh
Jean Guyomarc'h
Felipe Magno de Almeida
Vincent Torri
Chidambar Zinnoury