一、avro是一个数据序列化框架,可以高效得进行序列化和反序列化,支持C, C++, C#, Java, PHP, Python, 和Ruby语言。现在使用Java来读写。

二、环境搭建

  1、下载avro-1.7.7.jar and avro-tools-1.7.7.jar两个jar包,放到指定文件目录。下载地址 http://www.trieuvan.com/apache/avro/avro-1.7.7/java/

    我放到了D:\soft\avro 文件夹,在改目录下新建java文件夹,用来存放生成的Java代码

   2、该目录下新建user.avsc文件,内容是:  

{"namespace": "example.avro",
"type": "record",
"name": "User",
"fields": [
{"name": "name", "type": "string"},
{"name": "favorite_number", "type": ["int", "null"]},
{"name": "favorite_color", "type": ["string", "null"]}
 ]
}

  3、打开cmd,进入到该目录,执行命令生成User类,注意命令后面有个".",表示生成的代码放在本目录下。

java -jar avro-tools-1.7.7.jar compile schema user.avsc java .

  

  在该文件夹下的Java文件下的../example/avro/目录下就会生成User.java文件。

  4.使用eclipse新建maven项目,在pom.xml加入avro的依赖。  

<dependency>
<groupId>org.apache.avro</groupId>
<artifactId>avro</artifactId>
<version>1.7.7</version>
</dependency>

三、生成的User.java文件的内容如下。

  把生成的User.java类复制到工程中,注意这个User.java里面生成的User类及其内部类的包名默认是user.avsc文件中的namespace的值,

  在本例中也就是example.avro。需要全部替换为自己的包名。

  最简单的方法就是把User.java中的example.avro全部替换为自己的包名。

/**
* Autogenerated by Avro
*
* DO NOT EDIT DIRECTLY
*/
package example.avro;
@SuppressWarnings("all")
@org.apache.avro.specific.AvroGenerated
public class User extends org.apache.avro.specific.SpecificRecordBase implements org.apache.avro.specific.SpecificRecord {
public static final org.apache.avro.Schema SCHEMA$ = new org.apache.avro.Schema.Parser().parse("{\"type\":\"record\",\"name\":\"User\",\"namespace\":\"example.avro\",\"fields\":[{\"name\":\"name\",\"type\":\"string\"},{\"name\":\"favorite_number\",\"type\":[\"int\",\"null\"]},{\"name\":\"favorite_color\",\"type\":[\"string\",\"null\"]}]}");
public static org.apache.avro.Schema getClassSchema() { return SCHEMA$; }
@Deprecated public java.lang.CharSequence name;
@Deprecated public java.lang.Integer favorite_number;
@Deprecated public java.lang.CharSequence favorite_color; /**
* Default constructor. Note that this does not initialize fields
* to their default values from the schema. If that is desired then
* one should use <code>newBuilder()</code>.
*/
public User() {} /**
* All-args constructor.
*/
public User(java.lang.CharSequence name, java.lang.Integer favorite_number, java.lang.CharSequence favorite_color) {
this.name = name;
this.favorite_number = favorite_number;
this.favorite_color = favorite_color;
} public org.apache.avro.Schema getSchema() { return SCHEMA$; }
// Used by DatumWriter. Applications should not call.
public java.lang.Object get(int field$) {
switch (field$) {
case 0: return name;
case 1: return favorite_number;
case 2: return favorite_color;
default: throw new org.apache.avro.AvroRuntimeException("Bad index");
}
}
// Used by DatumReader. Applications should not call.
@SuppressWarnings(value="unchecked")
public void put(int field$, java.lang.Object value$) {
switch (field$) {
case 0: name = (java.lang.CharSequence)value$; break;
case 1: favorite_number = (java.lang.Integer)value$; break;
case 2: favorite_color = (java.lang.CharSequence)value$; break;
default: throw new org.apache.avro.AvroRuntimeException("Bad index");
}
} /**
* Gets the value of the 'name' field.
*/
public java.lang.CharSequence getName() {
return name;
} /**
* Sets the value of the 'name' field.
* @param value the value to set.
*/
public void setName(java.lang.CharSequence value) {
this.name = value;
} /**
* Gets the value of the 'favorite_number' field.
*/
public java.lang.Integer getFavoriteNumber() {
return favorite_number;
} /**
* Sets the value of the 'favorite_number' field.
* @param value the value to set.
*/
public void setFavoriteNumber(java.lang.Integer value) {
this.favorite_number = value;
} /**
* Gets the value of the 'favorite_color' field.
*/
public java.lang.CharSequence getFavoriteColor() {
return favorite_color;
} /**
* Sets the value of the 'favorite_color' field.
* @param value the value to set.
*/
public void setFavoriteColor(java.lang.CharSequence value) {
this.favorite_color = value;
} /** Creates a new User RecordBuilder */
public static example.avro.User.Builder newBuilder() {
return new example.avro.User.Builder();
} /** Creates a new User RecordBuilder by copying an existing Builder */
public static example.avro.User.Builder newBuilder(example.avro.User.Builder other) {
return new example.avro.User.Builder(other);
} /** Creates a new User RecordBuilder by copying an existing User instance */
public static example.avro.User.Builder newBuilder(example.avro.User other) {
return new example.avro.User.Builder(other);
} /**
* RecordBuilder for User instances.
*/
public static class Builder extends org.apache.avro.specific.SpecificRecordBuilderBase<User>
implements org.apache.avro.data.RecordBuilder<User> { private java.lang.CharSequence name;
private java.lang.Integer favorite_number;
private java.lang.CharSequence favorite_color; /** Creates a new Builder */
private Builder() {
super(example.avro.User.SCHEMA$);
} /** Creates a Builder by copying an existing Builder */
private Builder(example.avro.User.Builder other) {
super(other);
if (isValidValue(fields()[0], other.name)) {
this.name = data().deepCopy(fields()[0].schema(), other.name);
fieldSetFlags()[0] = true;
}
if (isValidValue(fields()[1], other.favorite_number)) {
this.favorite_number = data().deepCopy(fields()[1].schema(), other.favorite_number);
fieldSetFlags()[1] = true;
}
if (isValidValue(fields()[2], other.favorite_color)) {
this.favorite_color = data().deepCopy(fields()[2].schema(), other.favorite_color);
fieldSetFlags()[2] = true;
}
} /** Creates a Builder by copying an existing User instance */
private Builder(example.avro.User other) {
super(example.avro.User.SCHEMA$);
if (isValidValue(fields()[0], other.name)) {
this.name = data().deepCopy(fields()[0].schema(), other.name);
fieldSetFlags()[0] = true;
}
if (isValidValue(fields()[1], other.favorite_number)) {
this.favorite_number = data().deepCopy(fields()[1].schema(), other.favorite_number);
fieldSetFlags()[1] = true;
}
if (isValidValue(fields()[2], other.favorite_color)) {
this.favorite_color = data().deepCopy(fields()[2].schema(), other.favorite_color);
fieldSetFlags()[2] = true;
}
} /** Gets the value of the 'name' field */
public java.lang.CharSequence getName() {
return name;
} /** Sets the value of the 'name' field */
public example.avro.User.Builder setName(java.lang.CharSequence value) {
validate(fields()[0], value);
this.name = value;
fieldSetFlags()[0] = true;
return this;
} /** Checks whether the 'name' field has been set */
public boolean hasName() {
return fieldSetFlags()[0];
} /** Clears the value of the 'name' field */
public example.avro.User.Builder clearName() {
name = null;
fieldSetFlags()[0] = false;
return this;
} /** Gets the value of the 'favorite_number' field */
public java.lang.Integer getFavoriteNumber() {
return favorite_number;
} /** Sets the value of the 'favorite_number' field */
public example.avro.User.Builder setFavoriteNumber(java.lang.Integer value) {
validate(fields()[1], value);
this.favorite_number = value;
fieldSetFlags()[1] = true;
return this;
} /** Checks whether the 'favorite_number' field has been set */
public boolean hasFavoriteNumber() {
return fieldSetFlags()[1];
} /** Clears the value of the 'favorite_number' field */
public example.avro.User.Builder clearFavoriteNumber() {
favorite_number = null;
fieldSetFlags()[1] = false;
return this;
} /** Gets the value of the 'favorite_color' field */
public java.lang.CharSequence getFavoriteColor() {
return favorite_color;
} /** Sets the value of the 'favorite_color' field */
public example.avro.User.Builder setFavoriteColor(java.lang.CharSequence value) {
validate(fields()[2], value);
this.favorite_color = value;
fieldSetFlags()[2] = true;
return this;
} /** Checks whether the 'favorite_color' field has been set */
public boolean hasFavoriteColor() {
return fieldSetFlags()[2];
} /** Clears the value of the 'favorite_color' field */
public example.avro.User.Builder clearFavoriteColor() {
favorite_color = null;
fieldSetFlags()[2] = false;
return this;
} @Override
public User build() {
try {
User record = new User();
record.name = fieldSetFlags()[0] ? this.name : (java.lang.CharSequence) defaultValue(fields()[0]);
record.favorite_number = fieldSetFlags()[1] ? this.favorite_number : (java.lang.Integer) defaultValue(fields()[1]);
record.favorite_color = fieldSetFlags()[2] ? this.favorite_color : (java.lang.CharSequence) defaultValue(fields()[2]);
return record;
} catch (Exception e) {
throw new org.apache.avro.AvroRuntimeException(e);
}
}
}
}

四、用Java实现序列化,即写avro文件。

  新建一个Java类  

     public static void main(String[] args) throws IOException {
     // 声明并初始化User对象
     // 方式一
     User user1 = new User();
user1.setName("zhangsan");
user1.setFavoriteNumber(21);
user1.setFavoriteColor(null);

     // 方式二 使用构造函数
// Alternate constructor
User user2 = new User("Ben", 7, "red");
      
    // 方式三,使用Build方式
// Construct via builder
User user3 = User.newBuilder()
.setName("Charlie")
.setFavoriteColor("blue")
.setFavoriteNumber(null)
.build();
String path = "D:\\tmp\\user.avro"; // avro文件存放目录
DatumWriter<User> userDatumWriter = new SpecificDatumWriter<User>(User.class);
DataFileWriter<User> dataFileWriter = new DataFileWriter<User>(userDatumWriter);
dataFileWriter.create(user1.getSchema(), new File(path));
     // 把生成的user对象写入到avro文件
dataFileWriter.append(user1);
dataFileWriter.append(user2);
dataFileWriter.append(user3);
dataFileWriter.close();
}

  run一下代码,查看指定文件目录是否生成了avro文件。

五、Java读取avro文件,即实现avro反序列化。

public static void main(String[] args) throws IOException {
DatumReader<User> reader = new SpecificDatumReader<User>(User.class);
DataFileReader<User> dataFileReader = new DataFileReader<User>(new File("D:\\tmp\\user.avro"), reader);
User user = null;
while (dataFileReader.hasNext()) {
user = dataFileReader.next();
System.out.println(user);
}
}

运行结果:

{"name": "zhangsan", "favorite_number": 21, "favorite_color": null}
{"name": "Ben", "favorite_number": 7, "favorite_color": "red"}
{"name": "Charlie", "favorite_number": null, "favorite_color": "blue"}

至此,例子写完。

Java读写avro例子的更多相关文章

  1. java 读写文件例子

    在linux下可以读写中文 import java.io.*; import java.text.SimpleDateFormat; import java.util.*; public class ...

  2. java读写文件大全

     java读写文件大全 最初java是不支持对文本文件的处理的,为了弥补这个缺憾而引入了Reader和Writer两个类,这两个类都是抽象类,Writer中 write(char[] ch,int o ...

  3. Java读写文本文件操作

    package com.test; import java.io.BufferedReader; import java.io.BufferedWriter; import java.io.File; ...

  4. java 读写word java 动态写入 模板文件

    import java.io.ByteArrayOutputStream; import java.io.File; import java.io.FileInputStream; import ja ...

  5. Java读写文件方法总结

    Java读写文件方法总结 Java的读写文件方法在工作中相信有很多的用处的,本人在之前包括现在都在使用Java的读写文件方法来处理数据方面的输入输出,确实很方便.奈何我的记性实在是叫人着急,很多时候既 ...

  6. Java读写文件的几种方式

    自工作以后好久没有整理Java的基础知识了.趁有时间,整理一下Java文件操作的几种方式.无论哪种编程语言,文件读写操作时避免不了的一件事情,Java也不例外.Java读写文件一般是通过字节.字符和行 ...

  7. Java正则表达式匹配例子

    Java正则表达式匹配例子 package com.ibm.test; import java.util.regex.Matcher; import java.util.regex.Pattern; ...

  8. Java读写Windows共享文件夹 .

    版权声明:本文为博主原创文章,未经博主允许不得转载. 项目常常需要有访问共享文件夹的需求,例如共享文件夹存储照片.文件等.那么如何使用Java读写Windows共享文件夹呢? Java可以使用JCIF ...

  9. 使用java注解的例子有没有

    使用java注解的例子 参考文档:http://www.cnblogs.com/pepcod/archive/2013/02/20/2918719.html http://www.shaoqun.co ...

随机推荐

  1. 解决Kubernetes 1.7.3 kube-apiserver频繁异常重启的问题(转)

    原文的帖子无法访问,我只能粘贴内容 近期将之前的一个用Kubernetes 1.3.7的环境更换为最新发布的用kubeadm安装的Kubernetes 1.6.4 Dashboard无法访问的问题&g ...

  2. Kubernetes报错Failed to get system container stats for "/system.slice/kubelet.service"

    tail -f /var/log/message Nov 14 07:12:51 image kubelet: E1114 07:12:51.627782    3007 summary.go:92] ...

  3. 去除图像中的alpha通道或透明度

    自从appstore提交app改变后,虽然提交的流程还是和原来一样,但是相比以前还是有很大的改动,本来就不太喜欢 English,改版之后很多东西都变了,开发一个app就已经够他妈的蛋疼啦,上传一个a ...

  4. VMware上centos系统忘记账号和密码解决

    转载: https://blog.csdn.net/wyx_wyl/article/details/78246907 我们有时会忘记虚拟机上的登陆密码,今天来说一下怎样修改我们的密码.应该对大多数的l ...

  5. windows下pem转ppk

    下载puttygen.exe,启动 下载路径:链接:https://pan.baidu.com/s/1hstORTa 密码:kvfi pem -> ppk :通过PuTTYgen 转换 1. I ...

  6. 在Java中String类为什么要设计成final?String真的不可变吗?其他基本类型的包装类也是不可变的吗?

    最近突然被问到String为什么被设计为不可变,当时有点懵,这个问题一直像bug一样存在,竟然没有发现,没有思考到,在此总结一下. 1.String的不可变String类被final修饰,是不可继承和 ...

  7. springboot获取application.yml中的配置信息

    HelloController.java package com.springbootweb.demo.controller; import com.springbootweb.demo.entity ...

  8. keras训练和保存

    https://cloud.tencent.com/developer/article/1010815 8.更科学地模型训练与模型保存 filepath = 'model-ep{epoch:03d}- ...

  9. cocos源码分析--Sprite绘图原理

    精灵是2D游戏中最重要的元素,可以用来构成游戏中的元素,如人物,建筑等,用Sprite类表示,他将一张纹理的一部分或者全部矩形区域绘制到屏幕上.我们可以使用精灵表来减少OpenGL ES 绘制的次数, ...

  10. js中的substring

    "ABCDEFG".substring(2,3) 结果为"C"