protobuf protocol-buffers 序列化数据 gobs pickling string XML 用C实现的cPickle比pickle快1000倍 protobuf2 protobuf3 差异
场景:
浏览器请求--->python数据生成--->python-生成excel--->浏览器下载excel
目标:
重构为
浏览器请求--->python数据生成--->golang-生成excel--->浏览器下载excel
二阶目标:
后端全部golang实现
https://developers.google.com/protocol-buffers/
https://developers.google.com/protocol-buffers/docs/pythontutorial
Protocol Buffer Basics: Go | Protocol Buffers | Google Developers
https://developers.google.com/protocol-buffers/docs/gotutorial
一阶子探索:
1、python --- protocol-buffers --- golang
Protocol buffers are a language-neutral, platform-neutral extensible mechanism for serializing structured data.
Protocol Buffers - Google's data interchange format
Protocol Buffer Basics: Go | Protocol Buffers | Google Developers
https://developers.google.com/protocol-buffers/docs/gotutorial
Protocol Buffer Basics: Python | Protocol Buffers | Google Developers
https://developers.google.com/protocol-buffers/docs/pythontutorial
序列化和获取结构数据的方式
How do you serialize and retrieve structured data like this? There are a few ways to solve this problem:
- Use gobs to serialize Go data structures. This is a good solution in a Go-specific environment, but it doesn't work well if you need to share data with applications written for other platforms.
- You can invent an ad-hoc way to encode the data items into a single string – such as encoding 4 ints as "12:3:-23:67". This is a simple and flexible approach, although it does require writing one-off encoding and parsing code, and the parsing imposes a small run-time cost. This works best for encoding very simple data.
- Serialize the data to XML. This approach can be very attractive since XML is (sort of) human readable and there are binding libraries for lots of languages. This can be a good choice if you want to share data with other applications/projects. However, XML is notoriously space intensive, and encoding/decoding it can impose a huge performance penalty on applications. Also, navigating an XML DOM tree is considerably more complicated than navigating simple fields in a class normally would be.
Protocol buffers are the flexible, efficient, automated solution to solve exactly this problem. With protocol buffers, you write a .proto
description of the data structure you wish to store. From that, the protocol buffer compiler creates a class that implements automatic encoding and parsing of the protocol buffer data with an efficient binary format. The generated class provides getters and setters for the fields that make up a protocol buffer and takes care of the details of reading and writing the protocol buffer as a unit. Importantly, the protocol buffer format supports the idea of extending the format over time in such a way that the code can still read data encoded with the old format.
1/3、gobs -- golang pickling -- python
https://golang.org/pkg/encoding/gob/
Package gob manages streams of gobs - binary values exchanged between an Encoder (transmitter) and a Decoder (receiver). A typical use is transporting arguments and results of remote procedure calls (RPCs) such as those provided by package "net/rpc".
The implementation compiles a custom codec for each data type in the stream and is most efficient when a single Encoder is used to transmit a stream of values, amortizing the cost of compilation.
跨语言性差,局限在golang
Use Python pickling. This is the default approach since it's built into the language, but it doesn't deal well with schema evolution, and also doesn't work very well if you need to share data with applications written in C++ or Java.
11.1. pickle — Python object serialization — Python 2.7.16 documentation
https://docs.python.org/2/library/pickle.html
The pickle
module implements a fundamental, but powerful algorithm for serializing and de-serializing a Python object structure. “Pickling” is the process whereby a Python object hierarchy is converted into a byte stream, and “unpickling” is the inverse operation, whereby a byte stream is converted back into an object hierarchy. Pickling (and unpickling) is alternatively known as “serialization”, “marshalling,” [1] or “flattening”, however, to avoid confusion, the terms used here are “pickling” and “unpickling”.
This documentation describes both the pickle
module and the cPickle
module.
11.1. pickle — Python object serialization — Python 2.7.16 documentation
https://docs.python.org/2/library/pickle.html#module-cPickle
The cPickle
module supports serialization and de-serialization of Python objects, providing an interface and functionality nearly identical to the pickle
module. There are several differences, the most important being performance and subclassability.
First, cPickle
can be up to 1000 times faster than pickle
because the former is implemented in C. Second, in the cPickle
module the callables Pickler()
and Unpickler()
are functions, not classes. This means that you cannot use them to derive custom pickling and unpickling subclasses. Most applications have no need for this functionality and should benefit from the greatly improved performance of the cPickle
module.
The pickle data stream produced by pickle
and cPickle
are identical, so it is possible to use pickle
and cPickle
interchangeably with existing pickles. [10]
There are additional minor differences in API between cPickle
and pickle
, however for most applications, they are interchangeable. More documentation is provided in the pickle
module documentation, which includes a list of the documented differences.
2/3、string
普通字符串的缺点是只能描述简单的数据结构
3/3、XML
跨语言性好,但是资源消耗高,性能差
更小更快
Developer Guide | Protocol Buffers | Google Developers
https://developers.google.com/protocol-buffers/docs/overview
Developer Guide
Welcome to the developer documentation for protocol buffers – a language-neutral, platform-neutral, extensible way of serializing structured data for use in communications protocols, data storage, and more.
This documentation is aimed at Java, C++, or Python developers who want to use protocol buffers in their applications. This overview introduces protocol buffers and tells you what you need to do to get started – you can then go on to follow the tutorials or delve deeper into protocol buffer encoding. API reference documentation is also provided for all three languages, as well as language and style guides for writing .proto
files.
What are protocol buffers?
Protocol buffers are a flexible, efficient, automated mechanism for serializing structured data – think XML, but smaller, faster, and simpler. You define how you want your data to be structured once, then you can use special generated source code to easily write and read your structured data to and from a variety of data streams and using a variety of languages. You can even update your data structure without breaking deployed programs that are compiled against the "old" format.
How do they work?
You specify how you want the information you're serializing to be structured by defining protocol buffer message types in .proto
files. Each protocol buffer message is a small logical record of information, containing a series of name-value pairs. Here's a very basic example of a .proto
file that defines a message containing information about a person:
message Person {
required string name = 1;
required int32 id = 2;
optional string email = 3; enum PhoneType {
MOBILE = 0;
HOME = 1;
WORK = 2;
} message PhoneNumber {
required string number = 1;
optional PhoneType type = 2 [default = HOME];
} repeated PhoneNumber phone = 4;
}
As you can see, the message format is simple – each message type has one or more uniquely numbered fields, and each field has a name and a value type, where value types can be numbers (integer or floating-point), booleans, strings, raw bytes, or even (as in the example above) other protocol buffer message types, allowing you to structure your data hierarchically. You can specify optional fields, required fields, and repeated fields. You can find more information about writing .proto
files in the Protocol Buffer Language Guide.
Once you've defined your messages, you run the protocol buffer compiler for your application's language on your .proto
file to generate data access classes. These provide simple accessors for each field (like name()
and set_name()
) as well as methods to serialize/parse the whole structure to/from raw bytes – so, for instance, if your chosen language is C++, running the compiler on the above example will generate a class called Person
. You can then use this class in your application to populate, serialize, and retrieve Person
protocol buffer messages. You might then write some code like this:
Person person;
person.set_name("John Doe");
person.set_id(1234);
person.set_email("jdoe@example.com");
fstream output("myfile", ios::out | ios::binary);
person.SerializeToOstream(&output);
Then, later on, you could read your message back in:
fstream input("myfile", ios::in | ios::binary);
Person person;
person.ParseFromIstream(&input);
cout << "Name: " << person.name() << endl;
cout << "E-mail: " << person.email() << endl;
You can add new fields to your message formats without breaking backwards-compatibility; old binaries simply ignore the new field when parsing. So if you have a communications protocol that uses protocol buffers as its data format, you can extend your protocol without having to worry about breaking existing code.
You'll find a complete reference for using generated protocol buffer code in the API Reference section, and you can find out more about how protocol buffer messages are encoded in Protocol Buffer Encoding.
Why not just use XML?
Protocol buffers have many advantages over XML for serializing structured data. Protocol buffers:
- are simpler
- are 3 to 10 times smaller
- are 20 to 100 times faster
- are less ambiguous
- generate data access classes that are easier to use programmatically
For example, let's say you want to model a person
with a name
and an email
. In XML, you need to do:
<person>
<name>John Doe</name>
<email>jdoe@example.com</email>
</person>
while the corresponding protocol buffer message (in protocol buffer text format) is:
# Textual representation of a protocol buffer.
# This is *not* the binary format used on the wire.
person {
name: "John Doe"
email: "jdoe@example.com"
}
When this message is encoded to the protocol buffer binary format (the text format above is just a convenient human-readable representation for debugging and editing), it would probably be 28 bytes long and take around 100-200 nanoseconds to parse. The XML version is at least 69 bytes if you remove whitespace, and would take around 5,000-10,000 nanoseconds to parse.
Also, manipulating a protocol buffer is much easier:
cout << "Name: " << person.name() << endl;
cout << "E-mail: " << person.email() << endl;
Whereas with XML you would have to do something like:
cout << "Name: "
<< person.getElementsByTagName("name")->item(0)->innerText()
<< endl;
cout << "E-mail: "
<< person.getElementsByTagName("email")->item(0)->innerText()
<< endl;
However, protocol buffers are not always a better solution than XML – for instance, protocol buffers would not be a good way to model a text-based document with markup (e.g. HTML), since you cannot easily interleave structure with text. In addition, XML is human-readable and human-editable; protocol buffers, at least in their native format, are not. XML is also – to some extent – self-describing. A protocol buffer is only meaningful if you have the message definition (the .proto
file).
编译器
binary format <--- the text format
https://github.com/protocolbuffers/protobuf/releases/tag/v3.7.1
Protocol Buffers source code is hosted on GitHub: https://github.com/protocolbuffers/protobuf.
Our old Google Code repository is: https://code.google.com/p/protobuf/. We moved to GitHub on Aug 26, 2014 and no future changes will be made on the Google Code site. For latest code updates/issues, please visit our GitHub site.
Compiling Your Protocol Buffers
https://developers.google.com/protocol-buffers/docs/pythontutorial
protoc -I=$SRC_DIR --python_out=$DST_DIR $SRC_DIR/addressbook.proto
D:\pyPlusGlang\protoc-3.7.1-win64\bin>protoc -I=D:\pyPlusGlang\mysite\polls\ --python_out=D:\pyPlusG
lang\mysite\polls\ D:\pyPlusGlang\mysite\polls\addressbook.proto
D:\pyPlusGlang\mysite\polls\addressbook_pb2.py
# -*- coding: utf-8 -*-
# Generated by the protocol buffer compiler. DO NOT EDIT!
# source: addressbook.proto import sys
_b=sys.version_info[0]<3 and (lambda x:x) or (lambda x:x.encode('latin1'))
from google.protobuf import descriptor as _descriptor
from google.protobuf import message as _message
from google.protobuf import reflection as _reflection
from google.protobuf import symbol_database as _symbol_database
# @@protoc_insertion_point(imports) _sym_db = _symbol_database.Default() from google.protobuf import timestamp_pb2 as google_dot_protobuf_dot_timestamp__pb2 DESCRIPTOR = _descriptor.FileDescriptor(
name='addressbook.proto',
package='tutorial',
syntax='proto3',
serialized_options=_b('\n\024com.example.tutorialB\021AddressBookProtos\252\002$Google.Protobuf.Examples.AddressBook'),
serialized_pb=_b('\n\x11\x61\x64\x64ressbook.proto\x12\x08tutorial\x1a\x1fgoogle/protobuf/timestamp.proto\"\x87\x02\n\x06Person\x12\x0c\n\x04name\x18\x01 \x01(\t\x12\n\n\x02id\x18\x02 \x01(\x05\x12\r\n\x05\x65mail\x18\x03 \x01(\t\x12,\n\x06phones\x18\x04 \x03(\x0b\x32\x1c.tutorial.Person.PhoneNumber\x12\x30\n\x0clast_updated\x18\x05 \x01(\x0b\x32\x1a.google.protobuf.Timestamp\x1aG\n\x0bPhoneNumber\x12\x0e\n\x06number\x18\x01 \x01(\t\x12(\n\x04type\x18\x02 \x01(\x0e\x32\x1a.tutorial.Person.PhoneType\"+\n\tPhoneType\x12\n\n\x06MOBILE\x10\x00\x12\x08\n\x04HOME\x10\x01\x12\x08\n\x04WORK\x10\x02\"/\n\x0b\x41\x64\x64ressBook\x12 \n\x06people\x18\x01 \x03(\x0b\x32\x10.tutorial.PersonBP\n\x14\x63om.example.tutorialB\x11\x41\x64\x64ressBookProtos\xaa\x02$Google.Protobuf.Examples.AddressBookb\x06proto3')
,
dependencies=[google_dot_protobuf_dot_timestamp__pb2.DESCRIPTOR,]) _PERSON_PHONETYPE = _descriptor.EnumDescriptor(
name='PhoneType',
full_name='tutorial.Person.PhoneType',
filename=None,
file=DESCRIPTOR,
values=[
_descriptor.EnumValueDescriptor(
name='MOBILE', index=0, number=0,
serialized_options=None,
type=None),
_descriptor.EnumValueDescriptor(
name='HOME', index=1, number=1,
serialized_options=None,
type=None),
_descriptor.EnumValueDescriptor(
name='WORK', index=2, number=2,
serialized_options=None,
type=None),
],
containing_type=None,
serialized_options=None,
serialized_start=285,
serialized_end=328,
)
_sym_db.RegisterEnumDescriptor(_PERSON_PHONETYPE) _PERSON_PHONENUMBER = _descriptor.Descriptor(
name='PhoneNumber',
full_name='tutorial.Person.PhoneNumber',
filename=None,
file=DESCRIPTOR,
containing_type=None,
fields=[
_descriptor.FieldDescriptor(
name='number', full_name='tutorial.Person.PhoneNumber.number', index=0,
number=1, type=9, cpp_type=9, label=1,
has_default_value=False, default_value=_b("").decode('utf-8'),
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR),
_descriptor.FieldDescriptor(
name='type', full_name='tutorial.Person.PhoneNumber.type', index=1,
number=2, type=14, cpp_type=8, label=1,
has_default_value=False, default_value=0,
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR),
],
extensions=[
],
nested_types=[],
enum_types=[
],
serialized_options=None,
is_extendable=False,
syntax='proto3',
extension_ranges=[],
oneofs=[
],
serialized_start=212,
serialized_end=283,
) _PERSON = _descriptor.Descriptor(
name='Person',
full_name='tutorial.Person',
filename=None,
file=DESCRIPTOR,
containing_type=None,
fields=[
_descriptor.FieldDescriptor(
name='name', full_name='tutorial.Person.name', index=0,
number=1, type=9, cpp_type=9, label=1,
has_default_value=False, default_value=_b("").decode('utf-8'),
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR),
_descriptor.FieldDescriptor(
name='id', full_name='tutorial.Person.id', index=1,
number=2, type=5, cpp_type=1, label=1,
has_default_value=False, default_value=0,
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR),
_descriptor.FieldDescriptor(
name='email', full_name='tutorial.Person.email', index=2,
number=3, type=9, cpp_type=9, label=1,
has_default_value=False, default_value=_b("").decode('utf-8'),
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR),
_descriptor.FieldDescriptor(
name='phones', full_name='tutorial.Person.phones', index=3,
number=4, type=11, cpp_type=10, label=3,
has_default_value=False, default_value=[],
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR),
_descriptor.FieldDescriptor(
name='last_updated', full_name='tutorial.Person.last_updated', index=4,
number=5, type=11, cpp_type=10, label=1,
has_default_value=False, default_value=None,
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR),
],
extensions=[
],
nested_types=[_PERSON_PHONENUMBER, ],
enum_types=[
_PERSON_PHONETYPE,
],
serialized_options=None,
is_extendable=False,
syntax='proto3',
extension_ranges=[],
oneofs=[
],
serialized_start=65,
serialized_end=328,
) _ADDRESSBOOK = _descriptor.Descriptor(
name='AddressBook',
full_name='tutorial.AddressBook',
filename=None,
file=DESCRIPTOR,
containing_type=None,
fields=[
_descriptor.FieldDescriptor(
name='people', full_name='tutorial.AddressBook.people', index=0,
number=1, type=11, cpp_type=10, label=3,
has_default_value=False, default_value=[],
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR),
],
extensions=[
],
nested_types=[],
enum_types=[
],
serialized_options=None,
is_extendable=False,
syntax='proto3',
extension_ranges=[],
oneofs=[
],
serialized_start=330,
serialized_end=377,
) _PERSON_PHONENUMBER.fields_by_name['type'].enum_type = _PERSON_PHONETYPE
_PERSON_PHONENUMBER.containing_type = _PERSON
_PERSON.fields_by_name['phones'].message_type = _PERSON_PHONENUMBER
_PERSON.fields_by_name['last_updated'].message_type = google_dot_protobuf_dot_timestamp__pb2._TIMESTAMP
_PERSON_PHONETYPE.containing_type = _PERSON
_ADDRESSBOOK.fields_by_name['people'].message_type = _PERSON
DESCRIPTOR.message_types_by_name['Person'] = _PERSON
DESCRIPTOR.message_types_by_name['AddressBook'] = _ADDRESSBOOK
_sym_db.RegisterFileDescriptor(DESCRIPTOR) Person = _reflection.GeneratedProtocolMessageType('Person', (_message.Message,), dict( PhoneNumber = _reflection.GeneratedProtocolMessageType('PhoneNumber', (_message.Message,), dict(
DESCRIPTOR = _PERSON_PHONENUMBER,
__module__ = 'addressbook_pb2'
# @@protoc_insertion_point(class_scope:tutorial.Person.PhoneNumber)
))
,
DESCRIPTOR = _PERSON,
__module__ = 'addressbook_pb2'
# @@protoc_insertion_point(class_scope:tutorial.Person)
))
_sym_db.RegisterMessage(Person)
_sym_db.RegisterMessage(Person.PhoneNumber) AddressBook = _reflection.GeneratedProtocolMessageType('AddressBook', (_message.Message,), dict(
DESCRIPTOR = _ADDRESSBOOK,
__module__ = 'addressbook_pb2'
# @@protoc_insertion_point(class_scope:tutorial.AddressBook)
))
_sym_db.RegisterMessage(AddressBook) DESCRIPTOR._options = None
# @@protoc_insertion_point(module_scope)
https://developers.google.com/protocol-buffers/docs/pythontutorial
protobuf2 protobuf3 差异
addressbook.proto: Explicit default values are not allowed in proto3.
addressbook.proto: Required fields are not allowed in proto3.
addressbook.proto:8:12: Explicit 'optional' labels are disallowed in the Proto3 syntax. To define 'o
ptional' fields in Proto3, simply remove the 'optional' label, as fields are 'optional' by default.
Language Guide | Protocol Buffers | Google Developers
https://developers.google.com/protocol-buffers/docs/proto
读写消息 protobuf二进制文件
D:\pyPlusGlang\LProtocol-buffers>python pb2ReadingAMessage.py myADDRESS_BOOK_FILE
Person ID: 1
Name: a001
Can't test non-submessage field "Person.email" for presence in proto3.
Mobile phone #:
152001 D:\pyPlusGlang\LProtocol-buffers>python pb2WritingAMessage.py myADDRESS_BOOK_FILE
['pb2WritingAMessage.py', 'myADDRESS_BOOK_FILE']
Enter person ID number: 002
Enter name: a002
Enter email address (blank for none): b@qq.cn
Enter a phone number (or leave blank to finish): 142002
Is this a mobile, home, or work phone? home
Enter a phone number (or leave blank to finish): D:\pyPlusGlang\LProtocol-buffers>python pb2ReadingAMessage.py myADDRESS_BOOK_FILE
Person ID: 1
Name: a001
Can't test non-submessage field "Person.email" for presence in proto3.
Mobile phone #:
152001
Person ID: 2
Name: a002
Can't test non-submessage field "Person.email" for presence in proto3.
Home phone #:
142002 D:\pyPlusGlang\LProtocol-buffers>
address_book = addressbook_pb2.AddressBook()
写protobuf文件
# Add an address.
PromptForAddress(address_book.people.add())
# Write the new address book back to disk.
f = open(fname, "wb")
f.write(address_book.SerializeToString())
读protobuf文件
# Read the existing address book.
f = open(fname, "rb")
address_book.ParseFromString(f.read())
一阶
python生成的protobuf文件被golang读取 反序列化
生成golang
golang/protobuf: Go support for Google's protocol buffers
https://github.com/golang/protobuf
- Grab the code from the repository and install the
proto
package. The simplest way is to rungo get -u github.com/golang/protobuf/protoc-gen-go
. The compiler plugin,protoc-gen-go
, will be installed in$GOPATH/bin
unless$GOBIN
is set. It must be in your$PATH
for the protocol compiler,protoc
, to find it.
protoc -I=$SRC_DIR --go_out=$DST_DIR $SRC_DIR/addressbook.proto
protoc -I=D:\pyPlusGlang\LProtocol-buffers --go_out=D:\pyPlusGlang\LProtocol-buffers D:\pyPlusGlang\LProtocol-buffers\SearchRequest.proto
protoc.exe 需要protoc-gen-go.exe,会从当前目录开始查找
python
成功读写文件
D:\pyPlusGlang\LProtocol-buffers>python pb3ReadingAMessage.py sRqs_pb3
query: appleVSbanana
page_number: 1
result_per_page: 4 D:\pyPlusGlang\LProtocol-buffers>python pb3WritingAMessage.py sRqs_pb3
['pb3WritingAMessage.py', 'sRqs_pb3']
Enter query: aANDb
Enter page_number: 123
Enter result_per_page: 789 D:\pyPlusGlang\LProtocol-buffers>python pb3ReadingAMessage.py sRqs_pb3
query: appleVSbanana
page_number: 1
result_per_page: 4
query: aANDb
page_number: 123
result_per_page: 789 D:\pyPlusGlang\LProtocol-buffers>
syntax = "proto3"; message SearchRequest {
string query = 1;
int32 page_number = 2;
int32 result_per_page = 3;
} message SearchRequestList {
repeated SearchRequest sR = 1;
}
D:\pyPlusGlang\LProtocol-buffers\SearchRequestList_pb2.py
# -*- coding: utf-8 -*-
# Generated by the protocol buffer compiler. DO NOT EDIT!
# source: SearchRequestList.proto import sys
_b=sys.version_info[0]<3 and (lambda x:x) or (lambda x:x.encode('latin1'))
from google.protobuf import descriptor as _descriptor
from google.protobuf import message as _message
from google.protobuf import reflection as _reflection
from google.protobuf import symbol_database as _symbol_database
# @@protoc_insertion_point(imports) _sym_db = _symbol_database.Default() DESCRIPTOR = _descriptor.FileDescriptor(
name='SearchRequestList.proto',
package='',
syntax='proto3',
serialized_options=None,
serialized_pb=_b('\n\x17SearchRequestList.proto\"L\n\rSearchRequest\x12\r\n\x05query\x18\x01 \x01(\t\x12\x13\n\x0bpage_number\x18\x02 \x01(\x05\x12\x17\n\x0fresult_per_page\x18\x03 \x01(\x05\"/\n\x11SearchRequestList\x12\x1a\n\x02sR\x18\x01 \x03(\x0b\x32\x0e.SearchRequestb\x06proto3')
) _SEARCHREQUEST = _descriptor.Descriptor(
name='SearchRequest',
full_name='SearchRequest',
filename=None,
file=DESCRIPTOR,
containing_type=None,
fields=[
_descriptor.FieldDescriptor(
name='query', full_name='SearchRequest.query', index=0,
number=1, type=9, cpp_type=9, label=1,
has_default_value=False, default_value=_b("").decode('utf-8'),
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR),
_descriptor.FieldDescriptor(
name='page_number', full_name='SearchRequest.page_number', index=1,
number=2, type=5, cpp_type=1, label=1,
has_default_value=False, default_value=0,
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR),
_descriptor.FieldDescriptor(
name='result_per_page', full_name='SearchRequest.result_per_page', index=2,
number=3, type=5, cpp_type=1, label=1,
has_default_value=False, default_value=0,
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR),
],
extensions=[
],
nested_types=[],
enum_types=[
],
serialized_options=None,
is_extendable=False,
syntax='proto3',
extension_ranges=[],
oneofs=[
],
serialized_start=27,
serialized_end=103,
) _SEARCHREQUESTLIST = _descriptor.Descriptor(
name='SearchRequestList',
full_name='SearchRequestList',
filename=None,
file=DESCRIPTOR,
containing_type=None,
fields=[
_descriptor.FieldDescriptor(
name='sR', full_name='SearchRequestList.sR', index=0,
number=1, type=11, cpp_type=10, label=3,
has_default_value=False, default_value=[],
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR),
],
extensions=[
],
nested_types=[],
enum_types=[
],
serialized_options=None,
is_extendable=False,
syntax='proto3',
extension_ranges=[],
oneofs=[
],
serialized_start=105,
serialized_end=152,
) _SEARCHREQUESTLIST.fields_by_name['sR'].message_type = _SEARCHREQUEST
DESCRIPTOR.message_types_by_name['SearchRequest'] = _SEARCHREQUEST
DESCRIPTOR.message_types_by_name['SearchRequestList'] = _SEARCHREQUESTLIST
_sym_db.RegisterFileDescriptor(DESCRIPTOR) SearchRequest = _reflection.GeneratedProtocolMessageType('SearchRequest', (_message.Message,), dict(
DESCRIPTOR = _SEARCHREQUEST,
__module__ = 'SearchRequestList_pb2'
# @@protoc_insertion_point(class_scope:SearchRequest)
))
_sym_db.RegisterMessage(SearchRequest) SearchRequestList = _reflection.GeneratedProtocolMessageType('SearchRequestList', (_message.Message,), dict(
DESCRIPTOR = _SEARCHREQUESTLIST,
__module__ = 'SearchRequestList_pb2'
# @@protoc_insertion_point(class_scope:SearchRequestList)
))
_sym_db.RegisterMessage(SearchRequestList) # @@protoc_insertion_point(module_scope)
D:\pyPlusGlang\LProtocol-buffers\pb3WritingAMessage.py
#! /usr/bin/python
import SearchRequestList_pb2
import sys def PromptForSearchRequest(searchRequest):
searchRequest.query = input("Enter query: ")
searchRequest.page_number = int(input("Enter page_number: "))
searchRequest.result_per_page = int(input("Enter result_per_page: "))
print(sys.argv)
if len(sys.argv) != 2:
print("here is :", __file__, sys._getframe().f_lineno) # @print@ "Usage:", sys.argv[0], "SearchRequest__FILE"
sys.exit(-1) searchRequestList = SearchRequestList_pb2.SearchRequestList() # Read the existing SearchRequest book.
try:
f = open(sys.argv[1], "rb")
searchRequestList.ParseFromString(f.read())
f.close()
except IOError:
print("here is :", __file__,
sys._getframe().f_lineno) # @print@ sys.argv[1] + ": Could not open file. Creating a new one." # Add an SearchRequest.
PromptForSearchRequest(searchRequestList.sR.add()) # Write the new SearchRequest book back to disk.
f = open(sys.argv[1], "wb")
f.write(searchRequestList.SerializeToString())
f.close()
D:\pyPlusGlang\LProtocol-buffers\pb3ReadingAMessage.py
#! /usr/bin/python import SearchRequestList_pb2
import sys # Iterates though all people in the AddressBook and prints info about them.
def ListSr(searchRequestList):
for sR in searchRequestList.sR:
print("query:", sR.query)
print("page_number:", sR.page_number)
print("result_per_page:", sR.result_per_page) # Main procedure: Reads the entire address book from a file and prints all
# the information inside.
if len(sys.argv) != 2:
print("Usage:", sys.argv[0], "searchRequestList")
sys.exit(-1) searchRequestList = SearchRequestList_pb2.SearchRequestList() # Read the existing address book.
f = open(sys.argv[1], "rb")
searchRequestList.ParseFromString(f.read())
f.close() ListSr(searchRequestList)
protobuf protocol-buffers 序列化数据 gobs pickling string XML 用C实现的cPickle比pickle快1000倍 protobuf2 protobuf3 差异的更多相关文章
- protobuf Protocol Buffers 简介 案例 MD
Markdown版本笔记 我的GitHub首页 我的博客 我的微信 我的邮箱 MyAndroidBlogs baiqiantao baiqiantao bqt20094 baiqiantao@sina ...
- Xml,Json,Hessian,Protocol Buffers序列化对比
简介 这篇博客主要对Xml,Json,Hessian,Protocol Buffers的序列化和反序列化性能进行对比,Xml和Json的基本概念就不说了. Hessian:Hessian是一个轻量级的 ...
- protobuf(Protocol Buffers)java初体验
因为项目须要所以简单的研究了下protobuf.我也是參照网上的博客,所以大部分内容我也就不反复造轮子了.首先protobuf介绍点击这里,使用介绍点击这里,使用demo看这里. 我个人的第一个样例也 ...
- Protocol Buffers序列化原理
1. 使用Varint编码数据,越小的数据,用少的字节编码.如小于128的用一个字节编码,大于128的用多个字节编码.同时,每个字节最高为1或者0表示是否为数字的一部分. 2. 由于负数的补码表示很大 ...
- Hadoop基础-Protocol Buffers串行化与反串行化
Hadoop基础-Protocol Buffers串行化与反串行化 作者:尹正杰 版权声明:原创作品,谢绝转载!否则将追究法律责任. 我们之前学习过很多种序列化文件格式,比如python中的pickl ...
- ProtoBuf3语法指南(Protocol Buffers)_上
0.说明 ProtoBuf3语法指南, 又称为proto3, 是谷歌的Protocol Buffers第3个版本. 本文基于官方英文版本翻译, 加上了自己的理解少量修改, 一共分为上下两部分. 1.序 ...
- Netty中如何序列化数据
JDK提供了ObjectOutputStream和ObjectInputStream,用于通过网络对POJO的基本数据类型和图进行序列化和反序列化.该API并不复杂,而且可以被应用于任何实现了java ...
- Netty(五):Netty中如何序列化数据
JDK提供了ObjectOutputStream和ObjectInputStream,用于通过网络对POJO的基本数据类型和图进行序列化和反序列化.该API并不复杂,而且可以被应用于任何实现了java ...
- 使用纳米 Protocol buffers 作为序列化数据
使用纳米 Protocol buffers 作为序列化数据 Protocol Buffers 是 Google 公司开发的一种数据描述语言,类似于XML能够将结构化数据序列化. 但是它更小, 更快, ...
随机推荐
- ubuntu18.04 安装android studio
首先从官网下载android studio:Android Studio (安装前应先安装JDK环境) 得到android-studio-ide-191.5977832-linux.tar.gz 在安 ...
- c# List<Object>和List<实体>相互转化
开发的过程中总会遇到各种转化的问题,现在我做的开发接口中就遇到需要将List<Object> 中的Object又含有List<实体>归为一个list中,就是要list中没有Li ...
- JIT优化的小问题
同事问了个问题,挺有意思的,代码: public class TestJIT{ private static boolean sss; public static void main(String[] ...
- Winform工程反编译后的工作
Winform工程,反编译后,虽然能用,但不太好用. 因为form并没有像原生的那样. 所以,需要几个步聚: 1. 用ResGen工具,把二进制资源文件还原为xml格式: ResGen fromXX. ...
- OpenLayer3入门——[一]
一.OpenLayer3下载 首先下载OpenLayer3开发包,步骤如下: 下载地址https://github.com/openlayers/openlayers/releases/tag/v3. ...
- (一)AppScan的安装及破解
IBM AppScan是一款目前最好用的Web 应用安全测试工具,Rational AppScan 可自动化 Web 应用的安全漏洞评估工作,能扫描和检测所有常见的 Web 应用安全漏洞,例如 SQL ...
- Java8新特性--函数式编程
在jdk8中什么是函数式接口: 1.被@FunctionalInterface注解修饰的. 2.接口里边只有一个非default的方法. 满足以上2个条件的即为函数式接口,ps:即使一个接口没有被@F ...
- 什么是EDID,EDID能做什么,EDID基本介绍
转至 http://blog.csdn.net/Jkf40622/article/details/48311455 Q1: 为什么要写这篇文章? A1:在最近的工作中遇到了不少问题,其中很多都是和ED ...
- Codeforces Round #533 (Div. 2) E. Helping Hiasat(最大独立集)
题目链接:https://codeforces.com/contest/1105/problem/E 题意:有 n 个事件,op = 1 表示我可以修改昵称,op = 2 表示一个名为 s_i 的朋友 ...
- ES6遍历器 生成器 学习整理
遍历器[迭代器](Iterator) 就是这样一种机制.它是一种接口,为各种不同的数据结构提供统一的访问机制.任何数据结构只要部署 Iterator 接口,就可以完成遍历操作(即依次处理该数据结构的所 ...