Summary: This fixes especially the execution of edje_cc on Windows
Test Plan: execution of edje_cc
Reviewers: cedric, raster
Subscribers: #reviewers, #committers
Tags: #efl
Differential Revision: https://phab.enlightenment.org/D7834
Summary:
if the user or system attempts to cancel this thread then it should
stop blocking and exit in order to avoid potentially exiting after
efl has expected ecore-con to stop being active
@fix
fix T7041
Depends on D6354
Reviewers: ManMower, devilhorns
Reviewed By: ManMower
Subscribers: cedric, #committers
Tags: #efl
Maniphest Tasks: T7041
Differential Revision: https://phab.enlightenment.org/D6355
currently it's being defined in evil_fcntl.h, but the actual
implementation of fcntl() in evil_fcntl.c is causing problems with
sockets. So one possibility is to remove the ifdef, another is to
change the implementation.
so ecore_con/efl_net were using the standard ecore_thread thread pool
for doing things like dns lookups (that can take multiple minutes
until timeouts) and actual http transactions. similarly they can block
thread workers for long periods or indefinitely thus basically
blocking the whole eocre_thread pool and stopping others from sharing
it. the best solution we have right now is to bypass the thread pool
queue and have dedicated threads for these actions. what we should
have is a dedicated thread pool with each thread taking on N
connections (via select etc.) and the ability to create and destroy
thread pools for specific tasks so you can separate the work out from
other work. but that is basically a redesign of our thread pool infra
so let's do the quick solution here until that day comes.
this partially addresses D4640
a dedicated thread per image load though is going to be a lot nastier...
so here's the ugly problem. libproxy. yes. we've discussed memory
usage (e.g. it may have to execute javascript and pull in lots of deps
etc.) but we dlopene'd on the fly. ok... but this didn't solve another
issue i hit:
libproxy was causing enlightenment to abort(). some internal bit of
libproxy was raising a c++ exception. this wasn't caught. this causes
an abort(). takes down your entire desktop. FANTASTIC. this is bad. i
wouldnt' expect a library we depend on to be THIS anti-social but
libproxy seemingly is. it SHOULd catch its error sand just propagate
back to us so we can handle gracefully.
there reall is no way around this - isolate libproxy. it's even worse
that libproxy can load arbitrary modules that come from anywhere sho
who knows what issues this can cause. isolation is the best solution i
can think of.
so this makes an elf+net_proxy_helper we spawn the first time we need
a proxy lookup. we re-use that binary again and again until it exits
(it should exit after 10 seconds of being idle with no requests coming
in/pending). it'll respawn again later if needed. this involves now
the efl net threads having to marshall back to mainloop to do the
spawn and to write to the proxy process (reading is done by async exe
data events and the data is passed down a thread queue to the waitng
efl net thread). if the exe dies with pending requests unanswered then
it's respawned again and the req's are re-sent to it... just in case.
it has a limit on how often it'll respawn quickly.
this seems to work in my limited testing. this ALSO now isolates
memory usage of libproxy to another slave process AND this process
will die taking its memory with it once it's been idle for long
enough. that;s also another good solution to keeping libproxy impact
at bay.
These legacy API had the nasty behavior of keeping handles alive until
the pending events were dispatched, this could happen after the module
itself was shutdown, resulting in log to unregistered domains.
Then do not unregister the domain -- eina_shutdown will avoid leaks
anyway.
This is a major work and unfortunately couldn't be split into smaller
pieces as old code was highly coupled.
Ecore_Con_Server is now a wrapper around Efl_Net_Dialer_Simple
(ecore_con_server_connect()) and Efl_Net_Server_Simple
(ecore_con_server_add()), doing all that the original version did with
some fixes so ecore_con_ssl_server_upgrade() and
ecore_con_ssl_client_upgrade() are more usable -- see the examples and
-t/--type=tcp+ssl.
I tried to be bug-compatible, with code annotations where things
doesn't make sense. This was based on ecore_con_suite tests and some
manual experimenting with the examples, these can be helpful if you
find regressions (report/assign to me).
If we can parse the IP using inet_pton() and the port, there is no
reason to call getaddrinfo() in a thread.
This is required since ecore_con_suite (for ecore_con-over-efl_net) will
assume the server is running as soon as it's created.
windows is nasty and defines the value to be set or retrieved as
'char *', which triggers a warning when we use another kind of
pointer.
Partially addresses D4357.
windows is nasty and defines the payload to be sent or received as
'char *', which triggers a warning when we use another kind of
pointer.
Partially addresses D4357.
On Windows SOCKET is unsigned, thus will cause sign errors when
formatting with "%d" or comparing with signed values.
On UNIX it was quiet and easy to miss, thus a new #define can be used
to check for those. It will use 'unsigned long' as SOCKET, thus will
complain out loud and not even work correctly when using pointers on
64bits UNIX on mistakes -- which should improve the situation.
This helped to fix lots of missing conversions, all fixed.
This partially addresses D4357.
Clang raised a massive amount of warnings due to the struct sockaddr_un
not being declared before using it. So, include the header that declares
this structure first.
This introduces AF_UNIX server and dialer, these are not available on
Windows as in that platform we'll create a custom class for native
'local' communication.
In the future we can add a wrapper class Efl.Net.Local that will use
the class for each platform, but won't expose its details.
For instance, if we ever expose 'credentials' (which I didn't because
they are not portable), then it doesn't make sense to try to match
that on Windows. The 'Efl.Net.Local' would just stick to the basics:
Reader, Writer and Closer APIs.
This was a huge work, but now UDP is usable as seen in the examples.
Instead of relying on 'connect()', just do 'sendto()' and 'recvfrom()'
as they are universal. Multicast address can only be connected in
IPv4, IPv6 wasn't working and I'm not sure the IPv4 is portable to
other platforms.
Dialer will auto-join multicast groups is the dialed address is
one. Multicast properties such as time to live (hops) and loopback can
be configured. When joining multicast groups, the local
address/interface can be configured by 'IP@IFACE' format, with
'@IFACE' being optional.
Dialers will now auto-bind, so it can receive data as dialers are
expected to be bi-directional. One can manually specify the binding
address if there is such need.
Since datagrams must be read in their full size, otherwise the
remaining bits are dropped, expose next_datagram_size_query() in both
Efl.Net.Socket.Udp and Efl.Net.Server.Udp.Client.
To finalize UDP for real we need to introduce an 'Efl_Net_Ip_Address'
structure to serve as both IPv4 and IPv6 and expose 'sendto()' and
'recvfrom()'. These will come later as this commit is already too big.
Thanks to vtorri for poiting out about close() is not the correct
socket function, we should use closesocket() instead.
Also defined SOCKET to int on Linux so we can use the same 'type' and
avoid lots of ifdef in our code. On Windows it's unsigned, thus would
cause some warnings about incorrect signed comparison.
Defined INVALID_SOCKET=-1 and SOCKET_ERROR=-1 on non-Windows platforms
so we can keep the same construct 'function() == error' and it should
work on POSIX and windows.
I cannot test these on Windows, but the situation should be improved
with this commit.
if no hints were specified, getaddrinfo() will assume ai_flags as
AI_ADDRCONFIG | AI_V4MAPPED, which only reports useful results based
on what system supports. For instance AI_ADDRCONFIG will only return
IPv4 if IPv4 address exists, likewise IPv6 will only be returned if
IPv6 address is configured, avoiding these to be tried and error for
most local networks where such address could not be used. AI_V4MAPPED
will map IPv4 address over IPv6 if no IPv4 was found.
this allows nicer usage such as 'localhost:http' as the address, which
will resolve to [::1]:80 (if IPv6 is enabled) or 127.0.0.1:80 if only
IPv4 exists.
Like other toolkits, let's enable this automatically for users before
connecting to 255.255.255.255 IPv4 (IPADDR_BROADCAST), otherwise most
systems will just fail to connect and send packets.
libproxy allows various means to configure a proxy, will load from
gnome and kde configuration settings, envvars, macos and even windows
registry.
curl still doesn't use it, but we can make that later.
SOCKS is implemented in its own thread using synchronous/blocking
primitives, which simplifies the code a lot -- as well as simulate the
usage of Ecore_Thread as our users will likely do.
Since SOCKSv4a and SOCKSv5 allow name resolution, the whole
getaddrinfo() is done in the same thread, when needed, instead of a
separate thread to do that, which should also save some resources.
Instead of the legacy ECORE_CON_SOCKS_V4 and ECORE_CON_SOCKS_V5, now
we use socks_proxy, all_proxy and no_proxy. This matches our other
dialers http/websocket (which will use http_proxy, all_proxy and
no_proxy). If desired it's easy to add back support for those
variables, but I think we should just deprecate them. (The legacy code
will keep unchanged, thus direct users of ecore_con_server will still
use those -- just the previous users of ecore_con_server will be
converted to use the new API).
As discussed in the mailing list, many people will use worker threads
to execute blocking syscalls and mandating ecore_thread_check() for
voluntary preemption reduces the ecore_thread usefulness a lot.
A clear example is ecore_con usage of connect() and getaddrinfo() in
threads. If the connect timeout expires, the thread will be cancelled,
but it was blocked on syscalls and they will hang around for long
time. If the application exits, ecore will print an error saying it
can SEGV.
Then enable access to pthread_setcancelstate(PTHREAD_CANCEL_ENABLE)
via eina_thread_cancellable_set(EINA_TRUE), to pthread_cancel() via
eina_thread_cancel(), to pthread_cleanup_push()/pthread_cleanup_pop()
via EINA_THREAD_CLEANUP_PUSH()/EINA_THREAD_CLEANUP_POP() and so on.
Ecore threads will enforce non-cancellable threads on its own code,
but the user may decide to enable that and allow cancellation, that's
not an issue since ecore_thread now plays well and use cleanup
functions.
Ecore con connect/resolve make use of that and enable cancellable
state, efl_net_dialer_tcp benefits a lot from that.
A good comparison of the benefit is to run:
./src/examples/ecore/efl_io_copier_example tcp://google.com:1234 :stdout:
before and after. It will timeout after 30s and with this patch the
thread is gone, no ecore error is printed about possible SEGV.
both resolve (getaddrinfo()) and connect() are now done in
Ecore_Thread, avoid to block the main loop.
My plan is to always use the threaded connect() using a blocking
socket, only set it to non-blocking after the socket is returned to
the main thread and before it's accessible to the user. It will make
the connect behavior more uniform.
Some errors were moved from HTTP to Dialer as they are more generic.
It has been discussed on the ML (thread: "[RFC] rename efl_self") and
IRC, and has been decided we should rename it to this in order to avoid
confusion with the already established meaning of self which is very
similar to what we were using it for, but didn't have complete overlap.
Kudos to Marcel Hollerbach for initiating the discussion and
fighting for it until he convinced a significant mass. :)
This commit breaks API, and depending on compiler potentially ABI.
@feature
Efl.Object.event_callback_call no longer calls legacy smart callbacks;
calling only event callbacks registered with the given event description
pointer.
Create the method Efl.Object.event_callback_legacy_call to inherit the old
behavior from Efl.Object.event_callback_call, calling both Efl.Object events
and legacy smart callbacks.
Update all other files accordingly in order to still supply legacy
callbacks while they are necessary.
Efl.Net.Server defines how to accept new connections, doing the
bind(), listen() and accept() for protocols such as TCP.
Efl.Net.Dialer defines to to reach a server.
Both are based on Efl.Net.Socket as communication interface that is
based on Efl.Io.Reader, Efl.Io.Writer and Efl.Io.Closer, thus being
usable with code such as Efl.Io.Copier.
The Server will emit an event "client,add" with the established
Socket, which is a child and can be closed by both the server or the
user.
The Dialer extends the Socket and allows for creating one given an
address, that will be resolved and connected.
TCP is the initial implementation so we an validate the
interfaces. UDP, Unix-Local and SSL will come later as derivate
classes.
The examples are documented and should cover the basic principles:
- efl_io_copier_example can accept "tcp://IP:PORT" and will work as a
"netcat", can send data from socket, file or stdin to a socket,
file, stdout or stderr.
- efl_net_server_example listens for connections and can either reply
"Hello World!" and take some data or work as an echo-server,
looping back all received data to the user.
More complex interactions that require a "chat" between client and
server will be covered with new classes later, such as a queue that
empties itself once data is read.
Vincent Torri
Mike Blumenkrantz
Cedric BAIL
Jean-Philippe Andre
Pawel Aksiutowicz
Gustavo Sverzut Barbieri
Carsten Haitzler
Jean Guyomarc'h
Daniel Kolesa
Bruno Dilly
Tom Hacohen
Vitor Sousa