Skip to content

Commit c0c43a4

Browse files
committedFeb 4, 2016
content freshness
1 parent b7c1a08 commit c0c43a4

8 files changed

+23
-22
lines changed
 

‎articles/hdinsight/hdinsight-dotnet-avro-serialization.md

+8-7
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@
1414
ms.tgt_pltfrm="na"
1515
ms.devlang="na"
1616
ms.topic="article"
17-
ms.date="10/29/2015"
17+
ms.date="02/04/2015"
1818
ms.author="jgao"/>
1919

2020

@@ -32,7 +32,7 @@ The serialized representation of an object in the Avro system consists of two pa
3232
##The Hadoop scenario
3333
The Apache Avro serialization format is widely used in Azure HDInsight and other Apache Hadoop environments. Avro provides a convenient way to represent complex data structures within a Hadoop MapReduce job. The format of Avro files (Avro object container file) has been designed to support the distributed MapReduce programming model. The key feature that enables the distribution is that the files are “splittable” in the sense that one can seek any point in a file and start reading from a particular block.
3434

35-
##Serialization in the Microsoft Avro Library
35+
##Serialization in Avro Library
3636
The .NET Library for Avro supports two ways of serializing objects:
3737

3838
- **reflection** - The JSON schema for the types is automatically built from the data contract attributes of the .NET types to be serialized.
@@ -41,14 +41,16 @@ The .NET Library for Avro supports two ways of serializing objects:
4141
When the data schema is known to both the writer and reader of the stream, the data can be sent without its schema. In cases when an Avro object container file is used, the schema is stored within the file. Other parameters, such as the codec used for data compression, can be specified. These scenarios are outlined in more detail and illustrated in the code examples below.
4242

4343

44-
## Microsoft Avro Library prerequisites
44+
## Install Avro Library
45+
46+
The following are required before you install the libary:
4547

4648
- <a href="http://www.microsoft.com/download/details.aspx?id=17851" target="_blank">Microsoft .NET Framework 4</a>
4749
- <a href="http://james.newtonking.com/json" target="_blank">Newtonsoft Json.NET</a> (6.0.4 or later)
4850

4951
Note that the Newtonsoft.Json.dll dependency is downloaded automatically with the installation of the Microsoft Avro Library. The procedure for this is provided in the following section.
5052

51-
## Microsoft Avro Library installation
53+
5254
The Microsoft Avro Library is distributed as a NuGet package that can be installed from Visual Studio via the following procedure:
5355

5456
1. Select the **Project** tab -> **Manage NuGet Packages...**
@@ -59,11 +61,10 @@ Note that the Newtonsoft.Json.dll (>=6.0.4) dependency is also downloaded automa
5961

6062
You may want to visit the <a href="https://hadoopsdk.codeplex.com/wikipage?title=Avro%20Library" target="_blank">Microsoft Avro Library home page</a> to read the current release notes.
6163

62-
##Microsoft Avro Library source code
6364

6465
The Microsoft Avro Library source code is available at the <a href="https://hadoopsdk.codeplex.com/wikipage?title=Avro%20Library" target="_blank">Microsoft Avro Library home page</a>.
6566

66-
##Compiling the schema by using the Microsoft Avro Library
67+
##Compile schemas using Avro Library
6768

6869
The Microsoft Avro Library contains a code generation utility that allows creating C# types automatically based on the previously defined JSON schema. The code generation utility is not distributed as a binary executable, but can be easily built via the following procedure:
6970

@@ -90,7 +91,7 @@ Please note that namespaces are extracted from the JSON schema, using the logic
9091

9192
Microsoft.Hadoop.Avro.Tools codegen /i:C:\SDK\src\Microsoft.Hadoop.Avro.Tools\SampleJSON\SampleJSONSchema.avsc /o:. /nf:my.own.nspace
9293

93-
##<a name="samples"></a>Guide to the samples for the Microsoft Avro Library
94+
## Samples
9495
Six examples provided in this topic illustrate different scenarios supported by the Microsoft Avro Library. The Microsoft Avro Library is designed to work with any stream. In these examples, data is manipulated via memory streams rather than file streams or databases for simplicity and consistency. The approach taken in a production environment will depend on the exact scenario requirements, data source and volume, performance constraints, and other factors.
9596

9697
The first two examples show how to serialize and deserialize data into memory stream buffers by using reflection and generic records. The schema in these two cases is assumed to be shared between the readers and writers out-of-band.

‎articles/hdinsight/hdinsight-emulator-release-notes.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@
1313
ms.tgt_pltfrm="na"
1414
ms.devlang="na"
1515
ms.topic="article"
16-
ms.date="10/29/2015"
16+
ms.date="02/04/2016"
1717
ms.author="jgao"/>
1818

1919

‎articles/hdinsight/hdinsight-hadoop-collect-debug-heap-dumps.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@
1414
ms.tgt_pltfrm="na"
1515
ms.devlang="na"
1616
ms.topic="article"
17-
ms.date="11/12/2015"
17+
ms.date="02/04/2016"
1818
ms.author="jgao"/>
1919

2020

‎articles/hdinsight/hdinsight-hadoop-script-actions.md

+4-4
Original file line numberDiff line numberDiff line change
@@ -14,12 +14,12 @@
1414
ms.tgt_pltfrm="na"
1515
ms.devlang="na"
1616
ms.topic="article"
17-
ms.date="11/09/2015"
17+
ms.date="02/04/2016"
1818
ms.author="jgao"/>
1919

2020
# Develop Script Action scripts for HDInsight
2121

22-
Learn how to write Script Action scripts for HDInsight. For information on using Script Action scripts, see [Customize HDInsight clusters using Script Action](hdinsight-hadoop-customize-cluster.md). For the same article written for the HDInsight cluster on Linux operating system, see [Develop Script Action scripts for HDInsight](hdinsight-hadoop-script-actions-linux.md).
22+
Learn how to write Script Action scripts for HDInsight. For information on using Script Action scripts, see [Customize HDInsight clusters using Script Action](hdinsight-hadoop-customize-cluster.md). For the same article written for Linux-based HDInsight clusters, see [Develop Script Action scripts for HDInsight](hdinsight-hadoop-script-actions-linux.md).
2323

2424
Script Action can be used to install additional software running on a Hadoop cluster or to change the configuration of applications installed on a cluster. Script actions are scripts that run on the cluster nodes when HDInsight clusters are deployed, and they are executed once nodes in the cluster complete HDInsight configuration. A script action is executed under system admin account privileges and provides full access rights to the cluster nodes. Each cluster can be provided with a list of script actions to be executed in the order in which they are specified.
2525

@@ -30,7 +30,7 @@ Script Action can be used to install additional software running on a Hadoop clu
3030
3131
## Sample scripts
3232

33-
For provisioning HDInsight clusters on Windows operating system, the Script Action is Azure PowerShell script.The following is a sample script for configure the site configuration files:
33+
For creating HDInsight clusters on Windows operating system, the Script Action is Azure PowerShell script.The following is a sample script for configure the site configuration files:
3434

3535
param (
3636
[parameter(Mandatory)][string] $ConfigFileName,
@@ -214,7 +214,7 @@ or
214214

215215
### Throw exception for failed cluster deployment
216216

217-
If you want to get accurately notified of the fact that cluster customization did not succeed as expected, it is important to throw an exception and fail the cluster provisioning. For instance, you might want to process a file if it exists and handle the error case where the file does not exist. This would ensure that the script exits gracefully and the state of the cluster is correctly known. The following snippet gives an example of how to achieve this:
217+
If you want to get accurately notified of the fact that cluster customization did not succeed as expected, it is important to throw an exception and fail the cluster creation. For instance, you might want to process a file if it exists and handle the error case where the file does not exist. This would ensure that the script exits gracefully and the state of the cluster is correctly known. The following snippet gives an example of how to achieve this:
218218

219219
If(Test-Path($SomePath)) {
220220
#Process file in some way

‎articles/hdinsight/hdinsight-high-availability.md

+5-5
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@
1414
ms.tgt_pltfrm="na"
1515
ms.devlang="multiple"
1616
ms.topic="article"
17-
ms.date="10/29/2015"
17+
ms.date="02/04/2016"
1818
ms.author="jgao"/>
1919

2020

@@ -49,7 +49,7 @@ Standard implementations of Hadoop clusters typically have a single head node. H
4949

5050

5151

52-
## Check the active head node service status
52+
## Check active head node service status
5353
To determine which head node is active and to check on the status of the services running on that head node, you must connect to the Hadoop cluster by using the Remote Desktop Protocol (RDP). For the RDP instructions, see [Manage Hadoop clusters in HDInsight by using the Azure Portal](hdinsight-administer-use-management-portal.md#connect-to-hdinsight-clusters-by-using-rdp). Once you have remoted into the cluster, double-click on the **Hadoop Service Available ** icon located on the desktop to obtain status about which head node the Namenode, Jobtracker, Templeton, Oozieservice, Metastore, and Hiveserver2 services are running, or for HDI 3.0, the Namenode, Resource Manager, History Server, Templeton, Oozieservice, Metastore, and Hiveserver2 services.
5454

5555
![](./media/hdinsight-high-availability/Hadoop.Service.Availability.Status.png)
@@ -63,7 +63,7 @@ To access job logs on the secondary head node in the event that it has become th
6363
![](./media/hdinsight-high-availability/Hadoop.Head.Node.Log.Files.png)
6464

6565

66-
## Configure the size of the head node
66+
## Configure head node size
6767
The head nodes are allocated as large virtual machines (VMs) by default. This size is adequate for the management of most Hadoop jobs run on the cluster. But there are scenarios that may require extra-large VMs for the head nodes. One example is when the cluster has to manage a large number of small Oozie jobs.
6868

6969
Extra-large VMs can be configured by using either Azure PowerShell cmdlets or the HDInsight SDK.
@@ -100,8 +100,8 @@ For the SDK, the story is similar. The creation and provisioning of a cluster by
100100
};
101101

102102

103-
**References**
103+
## Next Steps
104104

105-
- [ZooKeeper](http://zookeeper.apache.org/ )
105+
- [Apache ZooKeeper](http://zookeeper.apache.org/ )
106106
- [Connect to HDInsight clusters using RDP](hdinsight-administer-use-management-portal.md#rdp)
107107
- [Using HDInsight .NET SDK](hdinsight-provision-clusters.md#sdk)

‎articles/hdinsight/hdinsight-learn-hadoop-use-sample-gallery.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@
1414
ms.tgt_pltfrm="na"
1515
ms.devlang="na"
1616
ms.topic="get-started-article"
17-
ms.date="10/29/2015"
17+
ms.date="02/04/2016"
1818
ms.author="jgao"/>
1919

2020
# Learn Hadoop by using the Azure HDInsight Getting Started Gallery

‎articles/hdinsight/hdinsight-run-samples.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@
1414
ms.tgt_pltfrm="na"
1515
ms.devlang="na"
1616
ms.topic="article"
17-
ms.date="10/29/2015"
17+
ms.date="02/04/2016"
1818
ms.author="jgao"/>
1919

2020
#Run Hadoop MapReduce samples in Windows-based HDInsight

‎articles/hdinsight/hdinsight-submit-hadoop-jobs-programmatically.md

+2-2
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@
1414
ms.tgt_pltfrm="na"
1515
ms.devlang="na"
1616
ms.topic="article"
17-
ms.date="10/30/2015"
17+
ms.date="02/04/2016"
1818
ms.author="jgao"/>
1919

2020
# Submit Hadoop jobs in HDInsight
@@ -27,7 +27,7 @@ Learn how to use Azure PowerShell to submit MapReduce and Hive jobs, and how to
2727
> - [Use Pig with HDInsight](hdinsight-use-pig.md)
2828
> - [Use MapReduce with HDInsight](hdinsight-use-mapreduce.md)
2929
30-
##Prerequisites
30+
###Prerequisites
3131

3232
Before you begin this article, you must have the following:
3333

0 commit comments

Comments
 (0)
Please sign in to comment.