彻底搞定阻塞非阻塞同步异步

所需积分/C币:14 2018-05-10 09:03:27 185KB PDF

本文档,是我本人翻译的一篇介绍linux系统同步异步阻塞非阻塞的知识的,你在网上看到的百分之八十的知识,可能都以讹传讹,让你看的一知半解,因为网上很多该类博客,没有讲解清楚,举得例子:如老王烧水,小王银行排队等,都是瞎扯。看完本文档,如果你还不明白,请私信我。或者给我发邮件:tccymm@hotmail.com 注:linux接口使用部分,我没有翻译,我觉得这部分应该自己做。文中很多括号部分的介绍,是我想让看的人更明白,从机器指令角度解释的。欢迎指点。
EDUcatlon Kernel ystem call- kernel context switch EAGAIN/ EWOULDBLOCK in tiale read a System call - ke mel contect sw itch CAGAIN/ EWOULDDLOCK Resd response System call -kernel context switc Data move ment fro ernel space to user space *****嗜 The implication of non-blocking is that an T/O command may not be satisfied immediately, requiring that the application make numerous calls to await completion. T his can bc cxtrcmcly incfficicnt bccausc in many cascs the application must busy-wait until the data is available or attcmpt to do other work while the command is performed in the kernel. As also shown in Figure 3, this method can introduce latency in the I/ o because any gap etween the data becoming available in the kernel and the user calling read to return it can reduce the overall data throughput 非阻塞的含乂是IO命令可能不会立即得到满足,要求应用程序进行多次调用以等待完成。这可能非常低效,因为在很多情况下,应月程 序必須忙于等待数据可用或在内核中执行合令时尝试执行其他工作。如图3所示,这种方法可能会在IO中引入延迟,因为内核中可用数据 与用户调用read以返回数据之间的任何间隙都会降低总体数据吞吐量。 >Asynchronous blocking 1/0 Another blocking paradigm is non-bloc king 1O with blocking notifications. In this modcl, non-bloc king 1/O is configured, and then the bloc king select system call is used to determine when there's any activity for an 1/0 descriptor. What makes the select call interesting is that it can be used to provide notification for not just one descriptor, but many. For each descriptor, you can request notification of the descriptor's ability to write data availability of read data, and also whether an error has occurred. 另一个阻寒模式是阳寒通知的非阻塞ⅠOε在此模型中,酊置了非阳塞IO,然后使用阳塞选择系统调用来确定何时有ⅠO描述符的活动。 使 select(系统调用,如 Iselect.pol.epol( ( linux))调用感兴趣的是,它可以用来为不仅一个摧述符提供遥知,而且可以提供很多通知。对于每 个描述符,可以请求通知描述符写入数据的能力,读取数据的可用性以及是否发生错误。 Figure 4. Typical flow of the asynchronous blocking 1/O model(select) 图4.异步阻塞ⅠO模犁的典型流稈(以 select调用为例) Ap pplIcation Kerne Re()口 ystem call-kernel cante t switc LAGAINCWOULDDLOC< initiate read I/o Read respon Select- data available(readable) System call -kernel contexl ww tch 口 a movemenIt fron Read () kernel space to user space The primary issue with the select call is that it's not very efficient. While it's a convenient model for asynchronous notific ation, its use for high- performance I'O is not advised select调用的主要问题是效率不高。虽然它是异步通知的便捷模式,但不建议将其用于高性能I/O >Asynchronous non-blocking I/O(AIO) Finally, the asynchronous non-blocking 1/0 modcl is onc of ovcrlapping proccssing with 1/0. The rcad rcqucst returns immcdiatcly, indicating that the read was successfully initiated. The application can then perform other process ing while the background read operation completes. When the read response arrives, a signal or a thread-based callback can be generated to complete the i o transaction 最后,异步非阻塞IO模型是与IO叠处理之一。read请求立即返回,表明read己成功启动。然后应用程序可以在后台read操作完成时执 行其他处理、当read响应到达时,可以生成信号或基于线程的回调来完成I/O事务, Figure 5. Typical flow of the asynchronous non-blocking I/O model 图5异步非阻塞IO模型的典型流程 Kerel aio_ read n Systcm call -kernel context switc initiate read vo Other processIng Data movement from kernel space to user Rcad respons space wilt i signal or callbeck processing 容*半农容半水水 The ability to overlap computation and l o processing in a single process for potentially multiple lo requests exploits the gap betw een processing speed and I/O speed. While one or more slow IO requests are pending, the CPU can perform other tasks or, more commonly, operate on already completed T/Os while other I/Os are initiated 在单个进程中重叠(将杋器指令和IO并行进行,计算指的是机器指令,也魷是程序代码编译后产生的机器码)计算和IO处理以头现潜在的 多个O请求的能力利用了处理速度和1O速度之间的差距。当一个或多个缓慢釣IO请求挂起时,CPU可以执行其他任务,或者更常见的 是,在启动其他IO时对己经完成的IO进行操作 The next section examines this model further, explores the apl, and then demonstrates a number of the commands 下一节将进一步检查这个模型,探索API,然后演示一些命令。 >Motivation for as ynchronous I/O 异步IO的动机 From the previous taxonomy of I/O models, you can see the motivation for AlO. The block ing models require the initiating application to block wher the lO has started. This means that it isn,'t possible to overlap processing and I/O at the same time. The synchronous non-blocking model allows overlap of processing and T/O, but it requires that the application check the status of the [o on a recurring basis. This leaves asynchronous non blocking 1/0, which permits overlap of processing and 1/O, including notific ation of 1/o completion 从以前的IO模型分类中,您可以看到AO的动机。阻塞模型要求启动应用程序在IO启动时阻塞。这意味着不可能同时重叠(将机器指令和 IO并行进行)处理(指的是机器指令,也就是程序代码编译后产生的机器码)和IO。同步卡阻塞模型允许重叠处理和IO,但它要求应用程 序循环检查IO的状态。这留下了异步非阻塞IO,这允许处理和IO的重叠,包拈IO完成的通知。 The functionality provided by the sclect function(asynchronous blocking 1/ O)is similar to AlO, cxccpt that it still blocks. However, it blocks on notific ations instead of the i o call select函数(异步阻塞IO)提供的功能与AIO类似,只是它仍然阻塞。但是,它会阻止通知而不是IO呼叫 >Introduction to alo for linux Io for linux AlO first entered the L inux kernel in 2. 5 and is now a standard feature of 2.6 production kernels AO首先在2.5版本中进入了Linu为核,现在是2.6版本内核的标准功能。 This section explores the asynchronous IO model for Linux to help you understand how to apply it in your applications 本节探讨Lix的异步IO模型,以帮助您了解如何将其应用丁您的应用程序。 In a traditional 17O modcl, there is an 1/O channel that is idcntificd by a unique handIc. In UNIX, thesc arc filc descriptors (w hich arc the same for files, pipes, sockets, and so on). In blocking I/O, you initiate a transfer and the system call returns when it's complete or an error has occurred 在传统的IO模型中,有一个由唯一句柄标识的IO通道。在UNX中,这些是文件描述符(对于文件,管道,套接字等是相同的)。在 阻塞I/O时,您启动传输,系统调用在完成或发生错误吋返冋。 about a transfer, including a user buffer for data. When notification for an 1/O occurs (called a completion), the aiocb structure is proved oo oo In asynchronous non-blocking I/O, you have the ability to initiate multiple transfers at the same time. This requires a unique context for each tra so you can identify it when it completes. In AlO, this is an aiocb(AlO I/O Control Block )structure. This structure contains all of the inform uniquely identify the completed I/O. The aPi demonstration shows how to do this 在异步非阻塞1O中,您可以同时启动多个传输。这需要每次传输都有一个独特的上下文,以便您在完成讨识別它。在AO中,这是一个 ocb( AIO I/O控制块)结构。此结构包含有关传输的所有信息,包括数据的用户缓冲区。当发生IO诓知(称为完成)时,提供aicb结 构来唯一标识已完成的IO。API演示展示了如何做到这一点。 AIO API The Alo interface API is quite simple, but it provides the necessary functions for data transfer w ith a couple of different notification models table 1 shows the AlO interface functions, which are further explained later in this section AIO接妾口API常简,但它提供了用几种不同的通知嫫型进行数据传输的必要功能。表1显示了AO接口功能,本节后面将进一步介绍 这些功能。 Table 1. Alo interface APis API Lunction Description aio read Request an asynchronous read operation Check the status of aio return get the return status of a completed asynchronous request onous operat aio suspend Suspend the calling process until one or more asynchronous requests have completed (or failed) aio cancel Cancel an asynchronous I/O request lio listio Initiate a list of I/O operations Each of these API functions uses the aiocb structure for initiating or checking. This structure has a number of elements, but Listing I shows only the ones that you'll need to (or can)use 这些AⅣ函数中的每一个都使用 aiocb结构来启动或检查。该结构有许多元素,但清单1只显示了您需要(或可以)使用的元素。 Listing 1. The aiocb structure show ing the relevant fields struct aiocb i int aio fildes / File Descriptor / Valid only for lio listio(r/w nop) volatile void *aio buf: //Data Buffer sie t aio nbytes /i Number of Bytes in Data Buffer struct sigevent aio sigevent; / Notification Structure / Internal fields * The sigevent structure tells Alo what to do when the lo completes. You'll explore this structure in the AlO demonstration. Now I'll show you how the individual API functions for AIO work and how you can use them The aio read function requests an asynchronous read operation for a valid file descriptor. The file descriptor can represent a file, a socket, or even a pipe The aio read function has the follow ing prototyp int aio read( struct aiocb*aiocbp) The aio rcad function returns immcdiatcly aftcr thc rcqucst has bccn qucucd. The return valuc is zero on success or-I on crror, where crrno is defined To perform a read, the application must initialize the aiocb structure. The follow ing short example illustrates filling in the aiocb request structure and using aio read to perform an async hronous read request (ignore notification for now ) It also shows use of the aio error function, but I'll explain that later Listing 2. Sample code for an asynchronous read with aio read finc lude <aio h> struct aiocb my aioch fd=open("file. txt",O RDONLY if(fd≤o)pa Zero out the aiocb structure (recommended " zero((char )&my aiocb, sizeof(struct aiocb)) / Alocate a data buffer for the aiocb request */ my aiocb aio buf= malloc bufsizei1) if my aiocb, aio buf) pcrror("malloc ) /* Initialize the necessary fields in the aiocb*/ my aioch ain fildes=fd; my aiocb aio nbytes=bufsize, my aiocb aio offset-0 ret=aio read( &my aiocb if (ret <0) perror("aio read); while( aio error( &my aiocb ) EINPROGRESS got rct bytes on the rcad *0) if((rel-aio return( &iny iocb ) g else i /* read failed consult errno */ Building with the alo interface You can find the function prototypes and other necessary symbolics in the aio. h header file. When building an application that uses this interface, you must use the PosiX real-time extensions library(librt) In Listing 2. after the file from whic h you're reading data is opened, you zero out your aiocb structure, and then allocate a data buffer. The reference to the data butfcr is placed into aio buf. Subscqucntly, you initialize the sizc of the buffer into aio nbytes. The aio offsct is sct to zero( thc first offset in the file). You set the file descriptor from w hich you're reading into aio fildes. After these fields are set, you call aio read to request the read. You can then make a call to aio error to determine the status of the aio read. As long as the status is EINPROGRESS, you busy-wait until the status changes. At this point, your request has either succeeded or failed. Note the similarities to reading from the file with the standard library functions. In addition to the asynchronous nature of aio read. another difference is setting the offset for the read. In a typical read call, the offset is maintained for you in the file descriptor context. For each read, the offset is updated so that subsequent reads address the next block of data. This isn' t possible w ith asynchronous I/o because you can perform many read requests simultaneously, so you must specify the offset for each partic ular read request aio crror The aio error function is used to determine the status of a request. Its prototype is int ain error( struct aiocb*aiocbp This function can return the follow ing EINPROGRESS, indicating the request has not yet completed ECANCELLED, indicating the request w as cancelled by the application I, indicating that an crror occurred for which you can consult crrno aio return Another difference between asynchronous I/O and standard blocking 1 O is that you don' t have inmediate access to the return status of your function hecause you're not bloc king on the read call. In a standard read call, the return status is provided upon return of the function. with asynchronous I/O, you use the aio return function. This function has the follow ing prototype ssize t aio return( struct aiocb"aiocbp This function is called only after the aio error call has determined that your request has completed (either successfully or in error). The return value of aio return is identical to that of the read or write system call in a synchronous context(number of bytes transferred or-I for error aio write The aio write function is used to request an asynchronous write. Its function prototype is int aio write( struct aiocb*aiocbp ) The aio write function returns immediately, indicating that the request has been enqueued (w ith a return of 0 on success and -1 on failure, with errno properly set) This is similar to the read system call, but one behavior difference is worth noting. Recall that the offset to be used is important with the read call However, with write, the offset is important only if used in a file context where the O APPENd option is not set. If O APPEND is set, then the offset is ignored and the data is appended to the end of the file. Otherwise, the aio offset field determines the offset at which the data is written to d You can use the aio suspend func tion to suspend (or block) the calling process until an asynchronous IO request has completed, a signal is raised, or an optional timeout occurs. ' The caller provides a list of aioc b rcfcrcnccs for w hich the completion of at least onc will causc aio suspend to return The function prototype for aio suspend is: int aio suspend( const struct aiocb*const clist[ int n, const struct timespec timeout Using aio suspend is quite simple. A list of aiocb references is provided. If any of them complete, the call returns with 0. Otherw ise, -l is returned indicating an error occurred. See Listing 3 Listing 3. Using the aio suspend function to block on asynchronous I/Os struct aioct*cbhst[MAX LIST /* Clear the list. *i zero( (char *)clist, sizeof(cblist)) / Toad one or more references into the list cbhist[o]=&my aiocb; ret= aio read( &my aiocb ); ct- aio suspend( clist, MAX LIST, NULL Note that the second argument of aio suspend is the number of elements in cblist, not the number of aiocb references. Any NULL element in the cblist is ignored by aio suspend If a timeout is provided to aio suspend and the timeout occurs, then- l is returned and errno contains eaGain aio cancel The aio cancel func tion allows you to cancel one or all outs tanding 1/O requests for a given file descriptor. Its prototype is int aio cance int fd, struct aiocbaioc bp To cancel a single request, provide the file descriptor and the aiocb reference. If the request is suc cessfully cancelled, the function returns AIO CANCELED. If the request completes, the function returns AIO NOTCANCELED To cancel all requests for a given file descriptor, provide that file descriptor and a null reference for aiocbp The function returns 10 CANCELED if all requests are canceled. AIO NOT CANCELED if at least one request couldnt be canceled, and AlO ALLDONE if none of the requests could be canceled. You can then evaluate each individual Alo request using aio error. If the request was canceled aio error returns-1 and errno is set to cAnceled Finally, AlO provides a way to initiate multiple transfers at the same time using the lio listio API function. This function is important because it means you can start lots of I/Os in the context of a single system call (meaning one kernel context switch). This is great from a performance perspective, so it's worth exploring. The lio listio API function has the follow ing prototype int lio listio( int mode, struct aiocb"list[, int nent struct sigcvcnt s1g The mode argument can be LIO WAIT or LIO NOWAIT. LIO WAIT blocks the call until all I/O has completed. LIO NOWAIT returns after the operations have been queued. The list is a list of aiocb references, w ith the max imum number of elements defined by nent. Note that elements of list may be null. w hich lio listio ignores. The sigevent reference defines the method for signal notification when all I/o is complete The request for lio listio is slightly different than the typical read or write request in that the operation must be specified. This is illustrated in listing 4. Listing 4. Using the lio listio function to initiate a list of rcqucst struct aiocb aiocb1. aiocb2 struct aiocb*list[MAX LIST]; , Prepare the first aiocb", aiocbl. aio fildes= fd aiocbl. aio buf = malloc( BUfSIZE+ aiocbI. aio nbytes= BUFSIZE aiocbl aio offset= next offset. iocbl. aio lio opcode=LIO rEAd; bero((char sizeof(list)) list[O]=&aiocbl list[1]--&aiocb2 ret=lio listio( LIO WAIT, list, MAX LIST, NULL The read operation is noted in the aio lio opcode field w ith LIO READ For a write operation, LIO WRITE is used but LIO NOP is also valid for no operationl >AlO notifications Now that you've seen the AlO functions that are available. this section digs into the methods that you can use for asynchronous notification. I'll explore asynchronous notification through signals and func tion callbacks 现在您已经看到了可用的AO函数,本节将深入介绍用于异步通知釣方法。我将通过信号和函教叵调来探索异步通知。 Asynchronous notification with signals 带信号的异步通知 The use of signals for interprocess communication(IPC)is a traditional mechanism in UNIX and is also supported by Alo. In this paradig, the application dcfincs a signal handler that is invoked when a spccifiod signal occurs. The apple ation then specifics that an asynchronous rcqucst will raise a signal when the request has completed. As part of the signal context, the particular aiocb request is provided to keep track of multiple potentially outstanding requests. Listing 5 demonstrates this notification method 使用信号进行井程间通信(IPC)是UNIX中的传统机制,也得到了AIO的支持。在这个范例中,应用程序定义了一个信号处理程序,它在 发生指定信号时衩调用。应用程序然后指定一个异步请求将在请求完成时发出一个信号。作为信号上下文的一部分,提供特定的 aitch请 求以跟踪多个潜在未决请求ε清单5演示了这种運知方法。 isting 5. Using signals as notification for AlO requests voK setup 10 Int struct sigaction sig act; struct aiocb my aiocb Set up the signal handler*/ sigemptyset(&sig act. sa mask sig act. sa flags- SA SIGINFO; sig act. sa sigaction- aio completion handler Set up the alo request bzcro((char *)&my aiocb, sizeof(struct aiocb)); my ajocb aio fildes= fd my aoc b aio buf= malloc(BUF SIZE+1); Imy aiocb aio nbytes- BUF SIZE my aiocb aio offsct= next offsct s Link the AIo request with the Signal Handler *, my aiocb aio sigevent sige notify= SIGEV SIGNAL; my aioch ain sigevent sige signo= SIGIO my aiocb aio sigevent sigev value sisal ptr=&my aiocb; / Map the Signal to the Signal Handler ret- sigaction( SIGIO, &sig act, NULL ret=aio read( &my aiocb void aio completion handler( int signo siginfo t info, void*context struct aioc / Ensure it's our signal */ if (info->si signo- SIGIO)& req-(struct aiocb *)info->si value. sival ptr 7 Did the request complete?s/ if(aio error( req )==0) / Request completed successfully, get the return status *. ret= aio return( req); In Listing 5, you set up your signal handler to catch the SIGIO signal in the aio completion handler function. You then initialize the aio sigevent structure to raise SIGIO for notification(w hich is specified via the SIGEV SIGNAL delinition in sige notify). When your read completes, your signal handler extracts the particular aiocb from the signals si value structure and checks the error status and return status to determine 1/o completion 在清单5屮,您设置了信号处理程序以捕获 aio completion handler函数屮的 SIGIO信号。然后初始化 aIo sigevent结枃以引发 SIGIO通知(通 过 sige notify中的 SIGEV SIGNAL定义指定)。读取完成后,号处理程序从信号的 si value结构中提取定的 aiocb,并检查错误状态和 返冋状态以确定I/O完成。 For performance, the completion handler is an ideal spot to continue the i o by requesting the next asynchronous transfer. In this way, when completion of one transfer has completed, you immediately start the next 对于性能而言,完成处理程序是通过诮求下一个异步传输来继续IO的理想位置。这样,当一次传输完成后,您立即开始下一步。 ynchronous notification with callbac ks 带回调的异步通知 An alternative notification mechanism is the system callback. Instead of raising a signal for notification, this mechanism calls a function in user-space Tor notification. You init ialize the aiocb relerence into the sigevent structure to uniquely identify the particular request being completed; see Listing 6. 另一种通知机制是系统回调。该机制不是提高通知的信号,而是调用用户空间中的函数来进行通知。您将 amoco引用初始化为 sigcvcnt结 构,以唯一标识正在完成的特定请求;见清单6。 Listing 6. Using thread callback notification for AlO requests void setup io(...) fd struct aiocb my aiocb; A Set up the Alo request/ zero((char )&my aiocb, sizeof(struct aiocb) my aioch ain fildes= fd

...展开详情
img
tccymm_1

关注 私信 TA的资源

上传资源赚积分,得勋章
    最新推荐