[简体中文版(推荐)](README_cn.md)
## Sogou C++ Workflow
[![License](https://img.shields.io/badge/License-Apache%202.0-green.svg)](https://github.com/sogou/workflow/blob/master/LICENSE)
[![Language](https://img.shields.io/badge/language-c++-red.svg)](https://en.cppreference.com/)
[![Platform](https://img.shields.io/badge/platform-linux%20%7C%20macos%20%7C%20windows-lightgrey.svg)](https://img.shields.io/badge/platform-linux%20%7C%20macos20%7C%20windows-lightgrey.svg)
[![Build Status](https://img.shields.io/github/workflow/status/sogou/workflow/ci%20build)](https://github.com/sogou/workflow/actions?query=workflow%3A%22ci+build%22++)
As **Sogou\`s C++ server engine**, Sogou C++ Workflow supports almost all **back-end C++ online services** of Sogou, including all search services, cloud input method, online advertisements, etc., handling more than **10 billion** requests every day. This is an **enterprise-level programming engine** in light and elegant design which can satisfy most C++ back-end development requirements.
#### You can use it:
* To quickly build an **HTTP server**:
~~~cpp
#include <stdio.h>
#include "workflow/WFHttpServer.h"
int main()
{
WFHttpServer server([](WFHttpTask *task) {
task->get_resp()->append_output_body("<html>Hello World!</html>");
});
if (server.start(8888) == 0) { // start server on port 8888
getchar(); // press "Enter" to end.
server.stop();
}
return 0;
}
~~~
* As a **multifunctional asynchronous client**, it currently supports `HTTP`, `Redis`, `MySQL` and `Kafka` protocols.
* To implement **client/server on user-defined protocol** and build your own **RPC system**.
* [srpc](https://github.com/sogou/srpc) is based on it and it is an independent open source project, which supports srpc, brpc, trpc and thrift protocols.
* To build **asynchronous workflow**; support common **series** and **parallel** structures, and also support any **DAG** structures.
* As a **parallel computing tool**. In addition to **networking tasks**, Sogou C++ Workflow also includes **the scheduling of computing tasks**. All types of tasks can be put into **the same** flow.
* As an **asynchronous file IO tool** in `Linux` system, with high performance exceeding any system call. Disk file IO is also a task.
* To realize any **high-performance** and **high-concurrency** back-end service with a very complex relationship between computing and networking.
* To build a **micro service** system.
* This project has built-in **service governance** and **load balancing** features.
* Wiki link : [PaaS Architecture](https://github.com/sogou/workflow/wiki)
#### Compiling and running environment
* This project supports `Linux`, `macOS`, `Windows`, `Android` and other operating systems.
* `Windows` version is currently released as an independent [branch](https://github.com/sogou/workflow/tree/windows), using `iocp` to implement asynchronous networking. All user interfaces are consistent with the `Linux` version.
* Supports all CPU platforms, including 32 or 64-bit `x86` processors, big-endian or little-endian `arm` processors, `loongson` processors.
* Master branch requires SSL and `OpenSSL 1.1` or above is recommended. Fully compatible with BoringSSL. If you don't like SSL, you may checkout the [nossl](https://github.com/sogou/workflow/tree/nossl) branch. But still need to link `crypto` for `md5` and `sha1`.
* Uses the `C++11` standard and therefore, it should be compiled with a compiler which supports `C++11`. Does not rely on `boost` or `asio`.
* No other dependencies. However, if you need `Kafka` protocol, some compression libraries should be installed, including `lz4`, `zstd` and `snappy`.
### Get started (Linux, macOS):
~~~sh
git clone https://github.com/sogou/workflow
cd workflow
make
cd tutorial
make
~~~~
### Get started (Debian Linux, ubuntu 22.04):
Sogou C++ Workflow has been packaged for Debian Linux and ubuntu 22.04.
To install the Workflow library for development purposes:
~~~~sh
sudo apt-get install libworkflow-dev
~~~~
To install the Workflow library for deployment:
~~~~sh
sudo apt-get install libworkflow1
~~~~
### Get stated (xmake)
If you want to use xmake to build workflow, you can see [xmake build document](docs/en/xmake.md)
# Tutorials
* Client
* [Creating your first task:wget](docs/en/tutorial-01-wget.md)
* [Implementing Redis set and get:redis\_cli](docs/en/tutorial-02-redis_cli.md)
* [More features about series:wget\_to\_redis](docs/en/tutorial-03-wget_to_redis.md)
* Server
* [First server:http\_echo\_server](docs/en/tutorial-04-http_echo_server.md)
* [Asynchronous server:http\_proxy](docs/en/tutorial-05-http_proxy.md)
* Parallel tasks and Series
* [A simple parallel wget:parallel\_wget](docs/en/tutorial-06-parallel_wget.md)
* Important topics
* [About error](docs/en/about-error.md)
* [About timeout](docs/en/about-timeout.md)
* [About global configuration](docs/en/about-config.md)
* [About DNS](docs/en/about-dns.md)
* [About exit](docs/en/about-exit.md)
* Computing tasks
* [Using the build-in algorithm factory:sort\_task](docs/en/tutorial-07-sort_task.md)
* [User-defined computing task:matrix\_multiply](docs/en/tutorial-08-matrix_multiply.md)
* [Use computing task in a simple way: go task](docs/en/about-go-task.md)
* Asynchronous File IO tasks
* [Http server with file IO:http\_file\_server](docs/en/tutorial-09-http_file_server.md)
* User-defined protocol
* [A simple user-defined protocol: client/server](docs/en/tutorial-10-user_defined_protocol.md)
* Timing tasks and counting tasks
* [About timer](docs/en/about-timer.md)
* [About counter](docs/en/about-counter.md)
* [About resource pool](docs/en/about-resource-pool.md)
* [About module](docs/en/about-module.md)
* Service governance
* [About service governance](docs/en/about-service-governance.md)
* [More documents about upstream](docs/en/about-upstream.md)
* Connection context
* [About connection context](docs/en/about-connection-context.md)
* Built-in clients
* [Asynchronous MySQL client:mysql\_cli](docs/en/tutorial-12-mysql_cli.md)
* [Asynchronous Kafka client: kafka\_cli](docs/en/tutorial-13-kafka_cli.md)
#### Programming paradigm
We believe that a typical back-end program=protocol+algorithm+workflow and should be developed completely independently.
* Protocol
* In most cases, users use built-in common network protocols, such as HTTP, Redis or various rpc.
* Users can also easily customize user-defined network protocol. In the customization, they only need to provide serialization and deserialization functions to define their own client/server.
* Algorithm
* In our design, the algorithm is a concept symmetrical to the protocol.
* If protocol call is rpc, then algorithm call is an apc (Async Procedure Call).
* We have provided some general algorithms, such as sort, merge, psort, reduce, which can be used directly.
* Compared with a user-defined protocol, a user-defined algorithm is much more common. Any complicated computation with clear boundaries should be packaged into an algorithm.
* Workflow
* Workflow is the actual business logic, which is to put the protocols and algorithms into the flow graph for use.
* The typical workflow is a closed series-parallel graph. Complex business logic may be a non-closed DAG.
* The workflow graph can be constructed directly or dynamically generated based on the results of each step. All tasks are executed asynchronously.
Basic task, task factory and complex task
* Our system contains six basic tasks: networking, file IO, CPU, GPU, timer, and counter.
* All tasks are generated by the task factory and automatically recycled after callback.
* Server task is one kind of special networking task, generated by the framework which calls the task factory, and handed over to the user through the process function.
* In most cases, the task generated by the user through the task factory is
没有合适的资源?快使用搜索试试~ 我知道了~
workflow源代码code
共380个文件
h:158个
cc:86个
md:62个
需积分: 7 0 下载量 124 浏览量
2023-01-18
22:40:11
上传
评论
收藏 608KB ZIP 举报
温馨提示
workflow源代码code
资源推荐
资源详情
资源评论
收起资源包目录
workflow源代码code (380个子文件)
BUILD 7KB
poller.c 30KB
kafka_parser.c 28KB
dns_parser.c 23KB
json_parser.c 19KB
http_parser.c 17KB
crc32c.c 15KB
mysql_parser.c 12KB
redis_parser.c 9KB
rbtree.c 9KB
thrdpool.c 5KB
msgqueue.c 4KB
bootstrap.c 3KB
mysql_stream.c 3KB
mpoller.c 2KB
mysql_byteorder.c 2KB
KafkaMessage.cc 82KB
Communicator.cc 39KB
WFKafkaClient.cc 39KB
WFConsulClient.cc 27KB
HttpTaskImpl.cc 21KB
KafkaTaskImpl.cc 20KB
WFGlobal.cc 18KB
UpstreamPolicies.cc 18KB
MySQLTaskImpl.cc 16KB
KafkaDataTypes.cc 15KB
upstream_unittest.cc 14KB
RedisMessage.cc 13KB
WFTaskFactory.cc 13KB
WFDnsResolver.cc 12KB
WFServiceGovernance.cc 12KB
RouteManager.cc 12KB
URIParser.cc 11KB
MySQLMessage.cc 10KB
FileTaskImpl.cc 9KB
HttpUtil.cc 9KB
uriparser_unittest.cc 9KB
CommScheduler.cc 9KB
IOService_linux.cc 9KB
MySQLResult.cc 8KB
HttpMessage.cc 8KB
tutorial-14-consul_cli.cc 7KB
task_unittest.cc 7KB
tutorial-12-mysql_cli.cc 7KB
http_unittest.cc 7KB
RedisTaskImpl.cc 7KB
UpstreamManager.cc 6KB
WFDnsClient.cc 6KB
tutorial-13-kafka_cli.cc 6KB
WFServer.cc 6KB
SSLWrapper.cc 5KB
Workflow.cc 5KB
tutorial-05-http_proxy.cc 5KB
IOService_thread.cc 5KB
tutorial-03-wget_to_redis.cc 4KB
StringUtil.cc 4KB
tutorial-09-http_file_server.cc 4KB
DnsTaskImpl.cc 4KB
tutorial-15-name_service.cc 4KB
redis_unittest.cc 4KB
DnsMessage.cc 4KB
tutorial-02-redis_cli.cc 4KB
tutorial-17-dns_cli.cc 4KB
tutorial-01-wget.cc 3KB
Executor.cc 3KB
tutorial-04-http_echo_server.cc 3KB
facilities_unittest.cc 3KB
DnsUtil.cc 3KB
tutorial-08-matrix_multiply.cc 3KB
client-uds.cc 3KB
message.cc 3KB
WFNameService.cc 3KB
client.cc 3KB
EncodeStream.cc 3KB
tutorial-06-parallel_wget.cc 3KB
DnsCache.cc 3KB
dns_unittest.cc 3KB
DnsRoutine.cc 3KB
tutorial-07-sort_task.cc 3KB
tutorial-11-graph_task.cc 3KB
WFResourcePool.cc 3KB
KafkaResult.cc 2KB
algo_unittest.cc 2KB
memory_unittest.cc 2KB
server-uds.cc 2KB
graph_unittest.cc 2KB
WFMessageQueue.cc 2KB
server.cc 2KB
WFGraphTask.cc 2KB
tutorial-20-reducer.cc 2KB
mysql_unittest.cc 2KB
resource_unittest.cc 2KB
WFMySQLServer.cc 2KB
MySQLUtil.cc 1KB
server.cc 1KB
benchmark-02-http_server_long_req.cc 1KB
PackageWrapper.cc 1KB
SubTask.cc 1KB
WFMySQLConnection.cc 1KB
benchmark-01-http_server.cc 1KB
共 380 条
- 1
- 2
- 3
- 4
资源评论
小沐~
- 粉丝: 14
- 资源: 8
上传资源 快速赚钱
- 我的内容管理 展开
- 我的资源 快来上传第一个资源
- 我的收益 登录查看自己的收益
- 我的积分 登录查看自己的积分
- 我的C币 登录后查看C币余额
- 我的收藏
- 我的下载
- 下载帮助
安全验证
文档复制为VIP权益,开通VIP直接复制
信息提交成功