Data Engineering with Apache Spark, Delta Lake, and Lakehouse : Create scalable pipelines that ingest, curate, and aggregate complex data in a timely and secure way
Language: English
Published by Packt Publishing, 2021
- Softcover
- New
Seller: AHA-BUCH GmbH, Einbeck, GermanyAHA-BUCH GmbH
AbeBooks seller since August 14, 2006
Condition: New
US$ 89.50
Quantity: 2 available
Add to basketItem description from seller
nach der Bestellung gedruckt Neuware - Printed after ordering - Understand the complexities of modern-day data engineering platforms and explore strategies to deal with them with the help of use case scenarios led by an industry expert in big dataKey Features:Become well-versed with the core concepts of Apache Spark and Delta Lake for building data platformsLearn how to ingest, process, and analyze data that can be later used for training machine learning modelsUnderstand how to operationalize data models in production using curated dataBook Description:In the world of ever-changing data and ever-evolving schemas, it is important to build data pipelines that can auto-adjust to changes. This book will help you build scalable data platforms that managers, data scientists, and data analysts can rely on.Starting with an introduction to data engineering, along with its key concepts and architectures, this book will show you how to use Microsoft Azure Cloud services effectively for data engineering. You'll cover data lake design patterns and the different stages through which the data needs to flow in a typical data lake. Once you've explored the main features of Delta Lake to build data lakes with fast performance and governance in mind, you'll advance to implementing the lambda architecture using Delta Lake. Packed with practical examples and code snippets, this book takes you through real-world examples based on production scenarios faced by the author in his 10 years of experience working with big data. Finally, you'll cover data lake deployment strategies that play an important role in provisioning the cloud resources and deploying the data pipelines in a repeatable and continuous way.By the end of this data engineering book, you'll have learned how to effectively deal with ever-changing data and create scalable data pipelines to streamline data science, ML, and artificial intelligence (AI) tasks.What You Will Learn:Discover the challenges you may face in the data engineering worldAdd ACID transactions to Apache Spark using Delta LakeUnderstand effective design strategies to build enterprise-grade data lakesExplore architectural and design patterns for building efficient data ingestion pipelinesOrchestrate a data pipeline for preprocessing data using Apache Spark and Delta Lake APIsAutomate deployment and monitoring of data pipelines in productionGet to grips with securing, monitoring, and managing data pipelines models efficientlyWho this book is for:This book is for aspiring data engineers and data analysts who are new to the world of data engineering and are looking for a practical guide to building scalable data platforms. If you already work with PySpark and want to use Delta Lake for data engineering, you'll find this book useful. Basic knowledge of Python, Spark, and SQL is expected.…
Seller Inventory # 9781801077743
- Title
- Data Engineering with Apache Spark, Delta Lake, and Lakehouse : Create scalable pipelines that ingest, curate, and aggregate complex data in a timely and secure way
- Author
- Manoj Kukreja
- Publisher
- Packt Publishing
- Publication year
- 2021
- Condition
- Neu
- Binding
- Taschenbuch
- Language
- English
- ISBN 10
- 1801077746
- ISBN 13
- 9781801077743
- Item weight
- 888 grams
- Dimensions
- 235x191x26 mm
Understand the complexities of modern-day data engineering platforms and explore strategies to deal with them with the help of use case scenarios led by an industry expert in big data
Key Features
- Become well-versed with the core concepts of Apache Spark and Delta Lake for building data platforms
- Learn how to ingest, process, and analyze data that can be later used for training machine learning models
- Understand how to operationalize data models in production using curated data
Book Description
In the world of ever-changing data and schemas, it is important to build data pipelines that can auto-adjust to changes. This book will help you build scalable data platforms that managers, data scientists, and data analysts can rely on.
Starting with an introduction to data engineering, along with its key concepts and architectures, this book will show you how to use Microsoft Azure Cloud services effectively for data engineering. You'll cover data lake design patterns and the different stages through which the data needs to flow in a typical data lake. Once you've explored the main features of Delta Lake to build data lakes with fast performance and governance in mind, you'll advance to implementing the lambda architecture using Delta Lake. Packed with practical examples and code snippets, this book takes you through real-world examples based on production scenarios faced by the author in his 10 years of experience working with big data. Finally, you'll cover data lake deployment strategies that play an important role in provisioning the cloud resources and deploying the data pipelines in a repeatable and continuous way.
By the end of this data engineering book, you'll know how to effectively deal with ever-changing data and create scalable data pipelines to streamline data science, ML, and artificial intelligence (AI) tasks.
What you will learn
- Discover the challenges you may face in the data engineering world
- Add ACID transactions to Apache Spark using Delta Lake
- Understand effective design strategies to build enterprise-grade data lakes
- Explore architectural and design patterns for building efficient data ingestion pipelines
- Orchestrate a data pipeline for preprocessing data using Apache Spark and Delta Lake APIs
- Automate deployment and monitoring of data pipelines in production
- Get to grips with securing, monitoring, and managing data pipelines models efficiently
Who this book is for
This book is for aspiring data engineers and data analysts who are new to the world of data engineering and are looking for a practical guide to building scalable data platforms. If you already work with PySpark and want to use Delta Lake for data engineering, you'll find this book useful. Basic knowledge of Python, Spark, and SQL is expected.
Table of Contents
- The Story of Data Engineering and Analytics
- Discovering Storage and Compute Data Lakes
- Data Engineering on Microsoft Azure
- Understanding Data Pipelines
- Data Collection Stage - The Bronze Layer
- Understanding Delta Lake
- Data Curation Stage - The Silver Layer
- Data Aggregation Stage - The Gold Layer
- Deploying and Monitoring Pipelines in Production
- Solving Data Engineering Challenges
- Infrastructure Provisioning
- Continuous Integration and Deployment (CI/CD) of Data Pipelines
"Synopsis" may belong to another edition of this title.
About the Author
"About the title" may belong to another edition of this title.
Shipping rates from Germany to U.S.A.
| Item | 7 to 10 business days | 5 to 7 business days |
|---|---|---|
| First item | US$ 39.27 | US$ 50.49 |
Payment methods
- Bank Wire Transfer
- Check
- Paypal
Store description
Das Unternehmen AHA-BUCH GmbH: Seit der Gründung von AHA-BUCH im Juli 2005 ist unser Hauptziel, zufriedenen Kunden so schnell und so preisgünstig wie möglich ihren Bücherwunsch zu erfüllen. Unsere Firma beschäftigt 16 Mitarbeiter, die nur ein Ziel kennen: den Kunden und seine Wünsche! Auf über 3700 m2 Fläche haben wir über 100.000 Bücher, Modernes Antiquariat und Spiele auf Lager.
Specialty
Kinderbücher & Kinderhör Casetten, German Books, Software, Natur & Tiere, Ratgeber, Sachbücher, Englische Bücher, Medizin & Gesundheit, Universität & StudiumSeller's business information
AHA-BUCH GmbH
Garlebsen 48
Einbeck, Germany 37574
Terms of sale
General Terms and Conditions and Customer Information / Privacy Policy
I. General Terms and Conditions
§ 1 Basic provisions
(1) The following terms and conditions apply to all contracts that you conclude with us as a provider (AHA-BUCH GmbH) via the Internet platforms AbeBooks and/or ZVAB. Unless otherwise agreed, the inclusion of any of your own terms and conditions used by you will be objected to
(2) A consumer within the meaning of the following regulations is any natural person who concludes a legal transaction for purposes that can predominantly be attributed neither to their commercial nor their independent professional activity. An entrepreneur is any natural or legal person or a partnership with legal capacity who, when concluding a legal transaction, acts in the exercise of his independent professional or commercial activity.
§ 2 Conclusion of the contract
(1) The object of the contract is the sale of goods.
(2) If an article is listed by us at AbeBooks or ZVAB, the activation of the offer page at AbeBooks or ZVAB constitutes the binding offer to conclude a contract under the conditions contained in the article page.
(3) The contract is concluded via the online shopping cart system as follows:
The goods intended for purchase are stored in the "shopping cart". Via the corresponding button in the navigation bar you can call up the "shopping cart" and make changes there at any time.
After calling up the "Checkout" page and entering the personal data as well as the payment and shipping conditions, all order data will be displayed again on the order overview page.
Before sending the order, you have the option of checking all information again, changing it (also via the "back" function of the Internet browser) or cancelling the purchase.
By submitting the order via the "Buy now" button, you declare the acceptance of the offer in a legally binding manner, whereby the contract is concluded.
(4) The processing of the order and transmission of all information required in connection with the conclusion of the contract is partly automated by e-mail. You must therefore ensure that the e-mail address you have stored with us is correct, that the receipt of the e-mails is technically ensured and, in particular, that spam filters do not prevent it.
§ 3 Right of retention, retention of title
(1) You can only exercise a right of retention if it concerns claims from the same contractual relationship.
(2) The goods remain our property until full payment of the purchase price.
§ 4 Warranty
(1) The statutory warranty rights exist.
(2) In the case of used goods, the warranty period is one year from delivery of the item, deviating from the statutory regulation. The shortening of the deadline does not apply:
- culpably caused damages attributable to us from injury to life, limb or health and in the case of other damages caused intentionally or through gross negligence;
- insofar as we have fraudulently concealed the defect or have assumed a guarantee for the quality of the item.
(3) As a consumer, you are requested to check the item immediately upon delivery for completeness, obvious defects and transport damage and to inform us and the freight forwarder of any complaints as soon as possible. If you do not comply with this, this will have no effect on your statutory warranty claims.
§ 5 Choice of law, place of performance, place of jurisdiction
(1) German law shall apply. In the case of consumers, this choice of law shall only apply insofar as this does not withdraw the protection afforded by mandatory provisions of the law of the state of the consumer's habitual residence (principle of favourability).
(2) The place of performance for all services arising from the existing business relationships with us as well as the place of jurisdiction is our registered office, insofar as you are not a consumer, but a merchant, a legal entity under public law or a special fund under public law. The same applies if you do not have a general place of jurisdiction in Germany or the EU or if the domicile or habitual residence is not known at the time the action is brought. The right to appeal to the court at another statutory place of jurisdiction remains unaffected by this.
(3) The provisions of the UN Convention on Contracts for the International Sale of Goods shall expressly not apply.
II. Customer Information
- Identity of the Seller
AHA-BUCH GmbH
Garlebsen 48
D-37574 Einbeck
Germany
Telephone: 055639996039
E-mail: abebooks@aha-buch.de.
Alternative dispute resolution:
The European Commission provides a platform for out-of-court online dispute resolution (ODR platform), which can be accessed under https://ec.europa.eu/odr.
- Information on the conclusion of the contract
The technical steps for the conclusion of the contract, the conclusion of the contract itself and the correction options are carried out in accordance with the provisions "Conclusion of the contract" of our General Terms and Conditions (Part I.).
- Contract language, contract text storage
3.1. The contract language is german.
3.2. The complete text of the contract will not be stored by us. Before sending the order, the contract data can be printed out or electronically saved via the print function of the browser. After receipt of the order by us, the order data, the legally prescribed information for distance contracts and the General Terms and Conditions will be sent to you again by e-mail.
- Essential characteristics of the goods or services
The essential characteristics of the goods and/or services can be found in the respective offer.
- Prices and terms of payment
5.1. The prices stated in the respective offers as well as the shipping costs represent total prices. They include all price components including all applicable taxes.
5.2. The shipping costs incurred are not included in the purchase price. They can be called up via a correspondingly designated button on our website or in the respective offer, are shown separately in the course of the ordering process and are to be borne by you in addition, unless free shipping has been promised.
5.3. If the delivery is made to countries outside the European Union, we may incur additional costs for which we are not responsible, such as.B customs duties, taxes or money transfer fees (transfer or exchange rate fees of the credit institutions), which are to be borne by you. Any costs incurred for the transfer of money shall also be borne by you in cases where the delivery is made to an EU member state, but the payment was initiated outside the European Union.
5.4. The payment methods available to you are indicated under a corresponding button on our website or in the respective offer.
5.5. Unless otherwise stated for the individual payment methods, the payment claims from the concluded contract are due for payment immediately.
- Terms of delivery
6.1. The terms of delivery, the delivery date and, if applicable, existing delivery restrictions can be found under a correspondingly designated button on our website or in the respective offer.
Insofar as no other deadline is specified in the respective offer or under the correspondingly designated button, the delivery of the goods will take place within 3-5 days after conclusion of the contract (in the case of agreed advance payment, however, only after the time of your payment instruction).
6.2. Insofar as you are a consumer, it is regulated by law that the risk of accidental loss and accidental deterioration of the sold item during dispatch shall only pass to you upon handover of the goods, regardless of whether the shipment is insured or uninsured. This does not apply if you have independently commissioned a transport company not named by the entrepreneur or a person otherwise designated to carry out the shipment.
- Statutory liability for defects
Liability for defects is governed by the "Warranty" provision in our General Terms and Conditions (Part I).
last update: 01/06/2022
Privacy policy
Unless otherwise stated below, the provision of your personal data is neither required by law or contract, nor is it necessary for the conclusion of a contract. You are not obliged to provide the data. Failure to provide this will have no consequences. This only applies if no other information is provided in the subsequent processing operations.
Collection, processing and disclosure of personal data when placing orders
When ordering, we collect and process your personal data only to the extent necessary to fulfil and process your order and to process your enquiries. The provision of the data is necessary for the conclusion of the contract. Failure to provide this provision means that no contract can be concluded. The processing takes place on the basis of Art. 6 para. 1 lit.b GDPR and is necessary for the performance of a contract with you.
Your data will be passed on, for example, to the shipping companies and dropshipping providers selected by you, payment service providers, service providers for order processing and IT service providers.
In all cases, we strictly observe the legal requirements. The scope of data transmission is limited to a minimum.
Duration of storage
After completion of the contract, the data will first be stored for the duration of the warranty period, then taking into account legal, in particular tax and commercial retention periods and then deleted after expiry of the period, unless you have consented to further processing and use.
Rights of the data subject
If the legal requirements are met, you have the following rights under Articles 15 to 20 GDPR: Right to information, to correction, to erasure, to restriction of processing, to data portability.
In addition, pursuant to Article 21 (1) GDPR, you have the right to object to processing based on Article 6 (1) f GDPR and to processing for direct marketing purposes.
Contact us on request. The contact details can be found in our imprint.
Right to lodge a complaint with the supervisory authority
In accordance with Article 77 GDPR, you have the right to complain to the supervisory authority if you believe that the processing of your personal data is not lawful.
Right to object
If the personal data processing listed here is based on our legitimate interest pursuant to Art. 6 para. 1 lit. f GDPR, you have the right to object to this processing at any time with effect for the future for reasons arising from your particular situation.
Shipping terms
We ship your order after we received them
for articles on hand latest 24 hours,
for articles with overnight supply latest 48 hours.
In case we need to order an article from our supplier our dispatch time depends on the reception date of the articles, but the articles will be shipped on the same day.
Our goal is to send the ordered articles in the fastest, but also most efficient and secure way to our customers.