High Performance in-memory computing with Apache Ignite.pdf

所需积分/C币:20 2018-06-19 10:47:22 17.75MB PDF

This book covers a verity of topics, including in-memory data grid, highly available service grid, streaming (event processing for IoT and fast data) and in-memory computing use cases from high-performance computing to get performance gains. The book will be particularly useful for those, who have t
Tweet This Book! Please help Shamim Ahmed Bhuiyan, Michael Zheludkov and Timur Isachenko by spreading the word about this book on twitter The suggested hashtag for this book is #shamim_ru Find out what other people are saying about the book by clicking on this link to search for this hashtag on Twitter https://twitter.com/search?q=#shamimru In memory of my Father.- Shamim Ahmed bhuiyan Contents Prca What this book covers Code samples about the authors 12345 Acknowledgments Introduction What is apache ignite? Modern application architecture with Apache Ignite Who uses apache ignite? Why Ignite instead of others Our Hope 3B4 Chapter one: Installation and the first Ignite application 15 Pre-requisites Installation Run multiple instances of Apache Ignite in a single host 18 Configure a multi-node cluster in different host Rest client to manipulate with the Apache Ignite 20 Java client SQL client Conclusion 32 What’ s Next Chapter two: Architecture overview 34 Functional overview 34 Cluster Topology Client and server Embedded with the application 37 Server in separate JVM(real cluster topology) Client and Server in separate JVM on single host Caching Topology .......... CONTENTS Partitioned caching topology d caching top 41 Local mode Caching strategy Cache-aside Read-through and Write-through 43 Write behind Data model CAP theorem and where does ignite stand in? Clustering 50 Cluster group Data collocation 52 Compute collocation with Data Zero spof How SoL queries works in ignite Multi-datacenter replication 56 Asynchronous support 58 Resilience Key APl 60 Conclusion 61 What's next 61 Chapter three: In-memory caching 62 Apache ignite as a 2nd level cache My Batis 2nd level cache Hibernate 2nd level cache Java method caching Web session clustering with Apache Ignite 96 Apache Ignite as a big memory, off-heap memory 10 Conclusion ...... 116 What's next 116 Chapter four: Persistence 117 Persistence Ignites cache 118 Persist RDBMS(PoStgreSQL) 121 Persistence in MongoDB 128 Cache queries ,135 Scan queries 137 SQL Projection and indexing with annotations 147 Query API 150 CONTENTS Collocated distributed joins 150 Non-collocated distributed joins 154 Performancc tuning SQL queries 156 Apache Ignite with JPA 156 Expiration Eviction of cache entries in Ignite 166 Expiration..... 166 Eviction..,,,,,,,,,,,, ···. ..170 Transaction 174 Ignite transac 176 Transaction commit protocol 178 Optimistic transactions 180 Pessimistic transactions 181 Performance impact on transaction 181 Conclusion ·· 182 What s next .182 Chapter five: Accelerating Big Data computing 183 Hadoop accelerator 183 In-memory Map/Reduce 185 Using Apache Pig for data analysis 197 Near real-time data analysis with Hive 204 Replace hdFS by Ignite In-memory File System (IGFS le system cache 225 gnite for Apache Spark A p ache spark an introducti 229 Ignite Context 231 IgnitcRDD 231 Preparing th e sandboX 232 Spark-shell to run Spark job park application example in Scala to share states 238 What's next 241 Chapter six: Streaming and complex event processing 242 Introducing data streamer gniteDataStreamer Strcamrecciver 250 Stream visitor 250 Camel data streamer 253 Direct Ingestion............ .,,255 260 Flume streamer Storm data streamer CONTENTS Conclusion 285 What's next 285 Chapter seven: Distributed computing 286 ompute gi Distributed closures MapReduce and Fork-join 295 Per-Node share state 303 Distributed task session Fault tolerance and checkpointing 313 Collocation of computation and data 323 Job scheduling 329 Service grid 331 Developing services 333 Cluster sir geton 338 Service management and configuration Developing microservices in Ignite ··: 352 Preface My first acquaintance with High load systems was at the beginning of 2007, and I started working on a real-world project since 2009. From that moment, I spent most of my office time with Cassandra, Hadoop, and numerous CEp tools. Our first Hadoop project(the year 2011-2012)with a cluster of 54 nodes often disappointed me with its long startup time. I have never been satisfied with the performance of our applications and was always looking for something new to boost the performance of our information systems. During this timc, I have tried HazclCast, Ehcache, Oracle Coherence as in-memory caches to gain the performance of the applications. I was usually disappointed from the complexity of using these libraries or from their functional limitations When I first encountered Apache Ignite, I was amazed It was the platform that Id been waiting on for a long time a simple spring based framework with a lot of awesome features such as DataBase caching, Big data acceleration, Streaming and compute/service grids In 2015, I had participated in Russian HighLoad++ conference with my presentation and started blogging in Dzone/Java Code Geeks and in my personal blog about developing High-load systems They became popular shortly, and i received a lot of feedback from the readers. Through them, I clarified the idea behind the book. The goal of the book was to provide a guide for those who really need to implement an in-memory platform in their projects. at the same time, the idea behind the book is not writing a manual. Although the Apache ignite platform is very big and growing day by day, we concentrate only on the features of the platform(from our point of view)that can really help to improve the performance of the applications We hope that High-performance in-memory computing with Apache ignite will be the go-to guide for architects and developers: both new and at an intermediate level, to get up and to develop with as little friction as possible Shamim ahmed What this book covers Introduction gives an overview of the trends that have made in-memory computing such important technology today. By the end of this chapter, you will have a clear idca of what Apache ignite arc and how can you design application with apache Ignite for getting maximum performance from your application Chapter one- Installation and the first Ignite application walks you through the initial setup of an Ignite grid and running of some sample application. At the end of the chapter, you will implement http://www.highload.ru/2015/abstracts/1875.html http://frommyworkshop.blogspot.ru 卩 reface your first simple Ignite application to read and write entries from the Cache. You will also learn how to install and configure an Sql idE to run SQL queries against Ignite caches Chapter two- Architecture overview covers the functional and architecture overview of the apache Ignite data fabrics. Here you will learn the concepts and the terminology of the apache ignite. This chapter introduces the main features of apache Ignite such as cluster topology, caching topology caching strategies, transactions, Ignite data model, data collocation and how SQL queries work Apache ignite. You will become familiar with some other concepts like multi-datacenter replication Ignite asynchronous support and resilience abilities Chapter three- In-memory caching presents some of the popular ignite data grid features, such as 2nd level cache, java method caching, web session clustering and off-heap memory. This chapter covers developments and technics to improve the performance of your existing web applications without changing any code Chapter four-Persistence guides you through the implementation of transactions and persistence of the Apache ignite cache. This chapter explores in depth: SQL feature and transaction of the Apache Ignite Chapter five- Accelerating Big Data computing, we focus on more advanced features and extensions to the Ignite platform. In this chapter, we will discuss the main problems of the Hadoop ecosystems and how Ignite can help to improve the performance of the exists Hadoop jobs. We detail the three main features of the Ignite Hadoop accelerator: in-memory Map/Reduce, IGFS, and Hadoop file system cache. We also provide examples of using Apache Pig and Hive to run Map/Reduce jobs on top of the Ignite in-memory Map/Reduce. At the end of the chapter, we show how to share states in-memory across different spark applications easily Chapter six Streaming and complex event processing takes the next step and goes beyond using Apache Ignite to solve complex real-time event processing problem. This chapter covers how ignite can be used easily with other Big Data technologies such as flume, storm, and camel to solve various business problems. We will guide you through with a few complete examples for developing real- time data processing on Apache Ignite Chapter seven-Distributive computing covers, how Ignite can help you to easily develop Microser vice like application, which will be performed in parallel fashion to gain high performance, low latency, and linear scalability. You will learn about Ignite MapReduce ForkJoin, Distributed closure execution, continuous mapping, ctc. for data processing across multiple nodes in the cluster Code samples A lI code samples, scripts, and more in-depth examples can be found on Github at GitHub repov https://github.com/srecon/ignite-book-code-sample

...展开详情
试读 127P High Performance in-memory computing with Apache Ignite.pdf
img
HyperJ

关注 私信 TA的资源

上传资源赚积分,得勋章
    最新推荐
    High Performance in-memory computing with Apache Ignite.pdf 20积分/C币 立即下载
    1/127
    High Performance in-memory computing with Apache Ignite.pdf第1页
    High Performance in-memory computing with Apache Ignite.pdf第2页
    High Performance in-memory computing with Apache Ignite.pdf第3页
    High Performance in-memory computing with Apache Ignite.pdf第4页
    High Performance in-memory computing with Apache Ignite.pdf第5页
    High Performance in-memory computing with Apache Ignite.pdf第6页
    High Performance in-memory computing with Apache Ignite.pdf第7页
    High Performance in-memory computing with Apache Ignite.pdf第8页
    High Performance in-memory computing with Apache Ignite.pdf第9页
    High Performance in-memory computing with Apache Ignite.pdf第10页
    High Performance in-memory computing with Apache Ignite.pdf第11页
    High Performance in-memory computing with Apache Ignite.pdf第12页
    High Performance in-memory computing with Apache Ignite.pdf第13页
    High Performance in-memory computing with Apache Ignite.pdf第14页
    High Performance in-memory computing with Apache Ignite.pdf第15页
    High Performance in-memory computing with Apache Ignite.pdf第16页
    High Performance in-memory computing with Apache Ignite.pdf第17页
    High Performance in-memory computing with Apache Ignite.pdf第18页
    High Performance in-memory computing with Apache Ignite.pdf第19页
    High Performance in-memory computing with Apache Ignite.pdf第20页

    试读已结束,剩余107页未读...

    20积分/C币 立即下载 >