The inspiration for writing this article was obtained after reading a similar publication for the x86 architecture [1].
This material will help those who want to understand how the programs are built from the inside, what happens before entering the main and why all this is done. Also I'll show you how to use some of the features of the glibc library. And in the end, as in the original article [1], the traversed path will be visually represented. Most of the article is a parsing of the glibc library.
So, let's start our trip. We will use Linux x86-64, and as a debugging tool - lldb. Also sometimes we will disassemble the program with objdump.
The source text is normal Hello, world (hello.cpp):
std::cout << "Hello, world!" << std::endl; |
Just in case, information about the system and programs * `uname -r` -- 4.12.10-1-ARCH |
clang++ -stdlib=libc++ hello1.cpp -g -o hello1.out |
The program using bash (and not only) is born by calling the fork function and creating a new process using execve, passing it the command-line arguments. Also, before the control of the first instruction of the executable file is transferred, the input and output descriptors (STDIN, STDOUT, STDERR) are set, then in the case of dynamic linking, libraries are loaded and initialized, and the functions of the ".preinit_array" section are called. Only after all this is called the first function, which is in the executable file (not counting the section ".preinit_array"), traditionally called _start, which is considered the beginning of the program. In the case of static linking, the work of the linker, for example, the initialization of the ".preinit_array" section, is inside the executable file and the functions themselves are slightly different from the dynamically linked programs. We will consider dynamically linked programs.
The entry point of the executable is specified in its header:
readelf -h hello1.out | grep Entry |
A little bit about ABI Definition of Wikipedia:
ABI (aplication binary interface) is a set of conventions for accessing an application to the operating system and other low-level services designed for portability of executable code between machines that have compatible ABIs. Unlike the API, which regulates compatibility at the source code level. ABI can be considered as a set of rules that allow the linker to combine the compiled component modules without recompiling all the code, while defining the binary interface.
The ABI level is hidden for c / c ++ programmers and all the work of this level is implemented by the compiler and the standard library libc. In my case, the clang compiler and the glibc library follow all ABI rules. ABI rules for Linux x86-64 are specified in the System V document AMD64 ABI [2]. Solaris, Linux, FreeBSD, OS X follow the conventions of this document. Microsoft has its own specific ABI, which they carefully hide. In the first chapter of this document [2] it is said that the architecture also obeys ABI rules for 32-bit processors [3]. Therefore, these are two basic documents on which developers of low-level libraries like glibc rely.
According to ABI, when the program starts, all registers are not defined except for:
% rdx: A pointer to a function that must be called before the program terminates.
% rsp: The stack is aligned on the 16-byte boundary, contains the number of arguments, the arguments themselves and the environment:
0(%rsp) argc
8(%rsp) argv[0]
...
8argc(%rsp) NULL
8(argc+1)(%rsp) envp[0]
...
8*(argc+k+1)(%rsp) envp[k]
NULL
auxiliary vectors
...
NULL
NULL
Auxiliary vectors (auxiliary vectors) contain information about the current machine. You can see their values using LD_SHOW_AUXV = 1 ./hello1.out |
And in fact is the number of program arguments
p * (char **) ($ rsp + 8) is the name of the program. Further on the stack are program arguments, zero delimiter, environment arguments and auxiliary vectors.
In addition, flags are set, SSE and x87 are configured (§3.4.1 [2]).
You can see that the arguments are already almost ready for the user-defined main function, only the right pointers are left. But in addition to setting up pointers before entering the main procedure, you need to do a lot more work. In the future, any function in its description will be accompanied by the location of its sources and the function itself in a binary form in the form of a tooltip, for example: main.
Let's look at the function _start, it's small and its main task is to transfer control of the __libc_start_main function.
We will disassemble the current function with the help of di (the output here and below is formatted for clarity):
lea 0x1aa(%rip), %r8 ; __libc_csu_fini |
lea 0x133(%rip), %rcx ; __libc_csu_init |
lea 0xec(%rip), %rdi ; main |
call *0x200796(%rip) ; __libc_start_main |
According to the ABI requirements, you need to zero% ebp to mark the frame as the initial one, which is done by the xor% ebp,% ebp instruction.
Next is the preparation for the call to the function __libc_start_main, the signature of which looks like this:
int __libc_start_main(int (*main) (int, char **, char **), |
void (*rtld_fini) (void), void *stack_end) |
| The |
The position for calling the |
Description |
| main |
%rdi |
The main function of the |
| argc |
%rsi |
The number of program arguments |
| argv |
%rdx |
Array of arguments. After the arguments are the environment variables, and afterwards the auxiliary vectors |
| init |
%rcx |
The global object constructor, called before main. The type of this function is the same as that of the main function. |
| fini |
%r8 |
The global object destructor, called after main |
| rtld_fini |
%r9 |
The destructor of the dynamic linker. Releases dynamically allocated libraries |
| stack_end |
%rsp |
The current position of the aligned stack |
ABI requires that when the function is called, the stack is aligned to 16-byte (sometimes 32, and sometimes 64, depending on the type of arguments) boundary. The request is executed after the execution of the instruction and $ -0x10,% rsp (?). The meaning of this alignment is that SIMD instructions (SSE, MMX) work only with aligned data, and scalar instructions faster read / are written with aligned data.
To store the 16-byte alignment, before registering __libc_start_main on the stack, the% rax register is stored in which an undefined value is stored. This stack cell will never be read.
The program should not return from the libc_start_main function, and to indicate the wrong behavior, use the hlt instruction. The peculiarity of this instruction is that in the protected mode of the processor it can be executed only in the protection ring 0, that is, only the operating system can call it. We are in the 3 ring, which means that when we try to execute a command that the program does not have rights to, we get a segmentation fault.
After the hlt instruction, there is another instruction nopl 0x0 (% rax,% rax, 1), which in turn is needed to align the next function with the 16-byte boundary. ABI does not require this, but compilers align the beginning of the function to improve performance ( 1 , 2 ).
So, let's go further
The source code of the __libc_start_main function shows that different code is generated for statically and dynamically linked libraries. You can check what the function code in the libc.so.6 library looks like with gdb or with lldb:
lldb libc.so.6 -b -o 'di -n __libc_start_main' |
# define __glibc_unlikely(cond) __builtin_expect ((cond), 0) |
# define __glibc_likely(cond) __builtin_expect ((cond), 1) |
The function __libc_start_main is a little cumbersome, it is enough to describe briefly its main actions:
registering rtld_fini with __cxa_atexit
call __libc_csu_init
create cancellation point
main
exit
__cxa_atexit
The function __cxa_atexit, in contrast to atexit, which is a wrapper over the first, can take parameters of the registered function, but the function should not call directly from the user space. It should not be called because the function uses a DSO identifier, which is known only to the compiler. It is needed so that when calling __cxa_atexit (f, p, d), the function f (p) is called when unloading DSO d [8].
However, passing arguments to the function-parameter Example of using __cxa_atexit:
extern "C" int __cxa_atexit (void (*func) (void *), void *arg, void *d); |
extern void* __dso_handle; |
int arg = *static_cast<int*>(a); |
__cxa_atexit(printArg, k, __dso_handle); |
rtld_fini is a pointer to the linker function _dl_fini. And yes, the linker is part of the glibc library. The _dl_fini function deinitializes and unloads all the loaded libraries.
__libc_csu_init
It is possible to get into the function __libc_csu_init in the same way as we got to the previous one. __libc_csu_init calls _init and function pointers in the .init_array section.
_init
The _init function is entirely in the .init section. Her code is divided into 2 parts: the introduction and the epilogue. The introduction consists of a prologue and an attempt to call the __gmon_start__ function.
leaq 0x105(%rip), %rax ; __gmon_start__ |
je 0x5555555548a2 ; je to addq instruction |
You can add your own code to the .init section. Consider the hello2.cpp example:
extern "C" void my_init() |
movq 0x200835(%rip), %rax |
callq 0x555555554990 ; ::my_init() |
An interesting fact: An attentive reader has probably noticed that output to hello2.cpp is output via the puts function. If you output via cout, then when compiling with the libstdc ++ library there will be a segmentation error, and with the libc ++ library the message will be displayed normally. Because of what does this happen? The matter is that in libstdc ++ cout it is initialized as the usual global object, and initialization of global objects occurs hardly later. In the case of libc ++, initialization occurs during the loading of libraries in the _dl_init function from the library ld-linux-x86-64.so.2. This function is just called from _dl_start_user just before passing control to the _start function.
Advantages and disadvantages of each method. When you connect the libc ++ library, even if you do not use the standard output from c ++ like cout, the constructors will in any case be called. In the case of the libstdc ++ library, even if the optimization flags are enabled, the constructor will be called as many times as the iostream header file is connected. Naturally, in the designer itself, the fact that it can be called several times and the re-initialization is skipped is taken into account. This, of course, will not slow down the initialization of the program, but it is still unpleasant. Apparently for this reason, many high-performance projects do not use, do not recommend, and even forbid to connect the iostream header file and, as a result, create their own interfaces for input-output.
.init_array
Further functions are called whose pointers are located in the .init_array section.
Let's check the contents of the section:
objdump hello1.out -s -j .init_array |
frame_dummy
Interestingly, frame_dummy is part of the gcc library.
What does gcc have to do with it? We have the same clang! compiler. Do not forget that the gcc project is very large and has already germinated into linux operating systems. The gcc project contains not only the compiler, but also the files needed for compilation. Thus, when linking, crt-files like crtbeginS.o and crtendS.o are used.
Therefore, completely it is not possible to get rid of the gcc project, and at least leave helper crt-files. Unix operating systems that do not use gcc as the main compiler do so.
frame_dummy looks like this:
jmp 0x555555554cc0 ; register_tm_clones |
The register_tm_clones function is needed in order to activate transactional the memory.
Initializing global objects
Global objects, if any, are initialized here.
In the presence of global objects, the address of the function _ GLOBAL__sub_I_ <compiled file name> |
Let's consider an example of initialization of global variables:
global1.cpp:
int k = printf("Hello from .init_array"); |
lea 0xf59(%rip), %rdi ; + 4 |
call 0x555555554e80 ; symbol stub for: printf |
mov %eax, 0x202130(%rip) ; k |
After calling printf, the result of the function is placed in the memory area of the variable k and an epilogue is called.
global2.cpp:
Suppose we have a certain class Global with a non-default constructor and destructor:
Then the initialization will look like this:
lea 0x202175(%rip), %rdi ; g |
call 0x5555555550e0 ; Global::Global() |
lea 0x1c5(%rip), %rdi ; Global::~Global() |
lea 0x202162(%rip), %rsi ; g |
lea 0x202147(%rip), %rdx ; __dso_handle |
call 0x555555554f10 ; symbol stub for: __cxa_atexit |
Initializing function call
From glibc, the initialization is invoked as follows: (* __ init_array_start [i]) (argc, argv, envp); |
In him, we can convey these arguments. Check the output of the program using the following global function:
void __attribute__((constructor)) hello(int argc, char **argv, char **env) |
printf("#args = %d\n", argc); |
printf("filename = %s\n", argv[0]); |
printf("Program has %d argument(s)\n", i); |
const C c(constructorArg); |
void __attribute__((constructor (65535))) hello(int argc, char ** argv, char **env){ |
As you probably already guessed, the program displays the correct number of arguments, and the most interesting, the object c is a constant. The main disadvantage of this approach is the lack of support for the standard and, as a consequence, the lack of cross-platform. Also, this code is highly dependent on the libc library in use.
I would like to add that global variables of the type int x = 1 + 2 * 3; they are not initialized at all, their values are initially written by the compiler into memory. If you want the variables initialized by simple functions likeint s = sum (4, 5) also to be initialized, add a constexpr identifier from the C ++ 11 standard to the sums function.
Creating cancellation point
The undo point is created by calling setjmp and setting the global variable.
Saving the context setjmp is necessary to set the cancel buffer, so that when canceling the main thread, it could be terminated correctly.
Example of canceling the main stream The file cancel.cpp.
pthread_t g_thr = pthread_self(); |
void * thread_start(void *) |
pthread_create(&thr, NULL, thread_start, NULL); |
You can verify that the context is really restored by putting a breakpoint immediately after the setjmp call:
br set -n __libc_start_main -R 162 |
Instead of the expected setjmp, the call takes place on __GI__set_mp. This function is the alias of the first and the creation of such an alias is made for each character used inside the library. This is done primarily to preserve the program's performance when replacing global characters and to increase performance [7]. Performance is improved by the fact that there is a direct call, and not through the PLT call table.
main
The source code of the function consists of one line.
std::cout << "Hello, world!" << std::endl; |
operator<<(std::cout, "Hello, World!").operator<<(std::endl); |
operator<<(std::cout, "Hello, World!"); |
std::cout.operator<<(std::endl); |
is a function both in libc ++ and in libstdc ++ and looks like this: ostream & endl (ostream &); |
Calling the first function first computes the length of the string. In my case, the length of the string is an IFUNC-symbol, that is, the implementation of the function is determined during program execution between __strlen_avx2 and _strlen_sse2. In your case, strlen is also possible without the use of vector registers.
When the data is written to stdout for the first time, the memory for the internal buffer in the _IO_file_doallocate function is allocated with malloc, and in my case the buffer size is 1 kb. File descriptors can have a different buffering policy , which is configured with setvbuf .
After the line is put, it is put into the stdout buffer, so the line is not immediately displayed. The second procedure adds a newline to the buffer and calls flush, which means that the internal bufferstdout is emptied and the drain is output to the screen.
In my case, after adding the end of the line, even without calling flush, control is passed to the function fwrite, which calls __libc_write, which calls syscall as follows (for clarity, the function is slightly simplified):
ssize_t __libc_write (int fd, const void *buf, size_t nbytes) |
unsigned long int resultvar = |
({ unsigned long int resultvar; |
long int __arg3 = (long int) (nbytes); |
long int __arg2 = (long int) (buf); |
long int __arg1 = (long int) (fd); |
register long int _a3 asm ("rdx") = __arg3; |
register long int _a2 asm ("rsi") = __arg2; |
register long int _a1 asm ("rdi") = __arg1; |
asm volatile ( "syscall\n\t" |
: "0" (1) , "r" (_a1), "r" (_a2), "r" (_a3) |
: "memory", "cc", "r11", "cx"); (long int) resultvar; }); |
int l = ({int b = 4; int c = 8; c += b}); |
The function __libc_write (in disassembled form called __GI___libc_write for the same reasons as _setjmp) for output to the console uses the system call interface syscall , and to call syscall, use the extension C language for assembler. The number of the system call is put in registerx. In the code, this is done using the architectural constraint on the used register = a, which means copying the result to the rax register, and "0" (1) in this case requires that before the script is executed, the value of 1 (sys_write) is in the register.
System calls, like ordinary calls, can have arguments, for example, for calling sys_write, the arguments are a file descriptor, a data buffer, and the size of the output text.
The kernel call conventions, according to ABI [2], differ from the usual call conventions. For the kernel, the list of parameters must lie in a certain order in the following registers:% rdi,% rsi,% rdx,% r10,% r8,% r9.
For architectures x86 and x86-64 system call tables are different !
When debugging, you might have noticed that before going into any function of the shared library from the main program for the first time, there are transitions and calls of various functions that we did not write. This is how the function addresses are defined using PIC technology ( 1 , 2 ).
exit
The exit function consists of the following calls:
__call_tls_dtors - causes thread local storage destructors, but in our case we did not use them.
Performs functions registered with atexit
_dl_fini is the same function that was passed to the_start in register9, which must be called before the program terminates.
Destructors of global objects (in our case they do not exist).
Calls the function of the __libc_atexit section
_IO_cleanup - clears and frees the file descriptor buffers.
_exit - terminates execution of all threads of the process.
The _exit function calls a system call 231 (sys_exit_group), which takes the return value of the program as a parameter in the% rdi register. This completes the program.
Linux also has a sys_exit system call. The difference between these two calls is that the latter terminates only the current thread, while sys_exit_group completes all threads of this process. In the case of a single-threaded process, the two data calls are equivalent, but in the case of a multi-threaded program, when the program terminates with sys_exit, the process will not complete threads and the system will not de-initialize the process until all its threads are completed [6].
This is the usual way that the processor goes through each time you run "Hello, World !!!", written in C / C ++, using the glibc library. Beyond the article, there are still a lot of things: the loader's work, the initialization of transactional memory, the implementation of the functions setjmp, atexit ...
It will be more obvious to show the done way in the form of a graph obtained with the help of dot
[1] - http:///tutorials/debugging/linuxProgramStartup.html
[2] - https://github.com/hjl-tools/ x86-psABI / wiki / x86-64-psABI-r252.pdf
[3] - https://github.com/hjl-tools/x86- psABI / wiki / intel386-psABI-1.1.pdf
[4] - https://dev./~vapier/crt.txt
[5] - http://syprog./2012/03/ linux-threads-through-magnifier-local.html
[6] - https:///glibc/wiki/Style_and_Conventions#Double-underscore_names_for_public_API_functions
[7] - https://itanium-cxx-abi./ cxx-abi / abi.html # dso-dtor
Papay
17 november 2017, 22:25
|
