场景:

浏览器请求--->python数据生成--->python-生成excel--->浏览器下载excel

目标:

重构为

浏览器请求--->python数据生成--->golang-生成excel--->浏览器下载excel

二阶目标:

后端全部golang实现

https://developers.google.com/protocol-buffers/

https://developers.google.com/protocol-buffers/docs/pythontutorial

Protocol Buffer Basics: Go  |  Protocol Buffers  |  Google Developers
https://developers.google.com/protocol-buffers/docs/gotutorial

一阶子探索:

1、python  --- protocol-buffers --- golang

Protocol buffers are a language-neutral, platform-neutral extensible mechanism for serializing structured data.

Protocol Buffers - Google's data interchange format

Protocol Buffer Basics: Go  |  Protocol Buffers  |  Google Developers
https://developers.google.com/protocol-buffers/docs/gotutorial

Protocol Buffer Basics: Python  |  Protocol Buffers  |  Google Developers
https://developers.google.com/protocol-buffers/docs/pythontutorial

序列化和获取结构数据的方式

How do you serialize and retrieve structured data like this? There are a few ways to solve this problem:

  • Use gobs to serialize Go data structures. This is a good solution in a Go-specific environment, but it doesn't work well if you need to share data with applications written for other platforms.
  • You can invent an ad-hoc way to encode the data items into a single string – such as encoding 4 ints as "12:3:-23:67". This is a simple and flexible approach, although it does require writing one-off encoding and parsing code, and the parsing imposes a small run-time cost. This works best for encoding very simple data.
  • Serialize the data to XML. This approach can be very attractive since XML is (sort of) human readable and there are binding libraries for lots of languages. This can be a good choice if you want to share data with other applications/projects. However, XML is notoriously space intensive, and encoding/decoding it can impose a huge performance penalty on applications. Also, navigating an XML DOM tree is considerably more complicated than navigating simple fields in a class normally would be.

Protocol buffers are the flexible, efficient, automated solution to solve exactly this problem. With protocol buffers, you write a .proto description of the data structure you wish to store. From that, the protocol buffer compiler creates a class that implements automatic encoding and parsing of the protocol buffer data with an efficient binary format. The generated class provides getters and setters for the fields that make up a protocol buffer and takes care of the details of reading and writing the protocol buffer as a unit. Importantly, the protocol buffer format supports the idea of extending the format over time in such a way that the code can still read data encoded with the old format.

1/3、gobs -- golang pickling -- python

https://golang.org/pkg/encoding/gob/

Package gob manages streams of gobs - binary values exchanged between an Encoder (transmitter) and a Decoder (receiver). A typical use is transporting arguments and results of remote procedure calls (RPCs) such as those provided by package "net/rpc".

The implementation compiles a custom codec for each data type in the stream and is most efficient when a single Encoder is used to transmit a stream of values, amortizing the cost of compilation.

跨语言性差,局限在golang

Use Python pickling. This is the default approach since it's built into the language, but it doesn't deal well with schema evolution, and also doesn't work very well if you need to share data with applications written in C++ or Java.

11.1. pickle — Python object serialization — Python 2.7.16 documentation
https://docs.python.org/2/library/pickle.html

The pickle module implements a fundamental, but powerful algorithm for serializing and de-serializing a Python object structure. “Pickling” is the process whereby a Python object hierarchy is converted into a byte stream, and “unpickling” is the inverse operation, whereby a byte stream is converted back into an object hierarchy. Pickling (and unpickling) is alternatively known as “serialization”, “marshalling,” [1] or “flattening”, however, to avoid confusion, the terms used here are “pickling” and “unpickling”.

This documentation describes both the pickle module and the cPickle module.

11.1. pickle — Python object serialization — Python 2.7.16 documentation
https://docs.python.org/2/library/pickle.html#module-cPickle

The cPickle module supports serialization and de-serialization of Python objects, providing an interface and functionality nearly identical to the pickle module. There are several differences, the most important being performance and subclassability.

First, cPickle can be up to 1000 times faster than pickle because the former is implemented in C. Second, in the cPickle module the callables Pickler() and Unpickler() are functions, not classes. This means that you cannot use them to derive custom pickling and unpickling subclasses. Most applications have no need for this functionality and should benefit from the greatly improved performance of the cPickle module.

The pickle data stream produced by pickle and cPickle are identical, so it is possible to use pickle and cPickle interchangeably with existing pickles. [10]

There are additional minor differences in API between cPickle and pickle, however for most applications, they are interchangeable. More documentation is provided in the pickle module documentation, which includes a list of the documented differences.

2/3、string

普通字符串的缺点是只能描述简单的数据结构

3/3、XML

跨语言性好,但是资源消耗高,性能差

更小更快

Developer Guide  |  Protocol Buffers  |  Google Developers
https://developers.google.com/protocol-buffers/docs/overview

 

Developer Guide

Welcome to the developer documentation for protocol buffers – a language-neutral, platform-neutral, extensible way of serializing structured data for use in communications protocols, data storage, and more.

This documentation is aimed at Java, C++, or Python developers who want to use protocol buffers in their applications. This overview introduces protocol buffers and tells you what you need to do to get started – you can then go on to follow the tutorials or delve deeper into protocol buffer encoding. API reference documentation is also provided for all three languages, as well as language and style guides for writing .proto files.

What are protocol buffers?

Protocol buffers are a flexible, efficient, automated mechanism for serializing structured data – think XML, but smaller, faster, and simpler. You define how you want your data to be structured once, then you can use special generated source code to easily write and read your structured data to and from a variety of data streams and using a variety of languages. You can even update your data structure without breaking deployed programs that are compiled against the "old" format.

How do they work?

You specify how you want the information you're serializing to be structured by defining protocol buffer message types in .proto files. Each protocol buffer message is a small logical record of information, containing a series of name-value pairs. Here's a very basic example of a .proto file that defines a message containing information about a person:

 
  1. message Person {
  2. required string name = 1;
  3. required int32 id = 2;
  4. optional string email = 3;
  5.  
  6. enum PhoneType {
  7. MOBILE = 0;
  8. HOME = 1;
  9. WORK = 2;
  10. }
  11.  
  12. message PhoneNumber {
  13. required string number = 1;
  14. optional PhoneType type = 2 [default = HOME];
  15. }
  16.  
  17. repeated PhoneNumber phone = 4;
  18. }

As you can see, the message format is simple – each message type has one or more uniquely numbered fields, and each field has a name and a value type, where value types can be numbers (integer or floating-point), booleans, strings, raw bytes, or even (as in the example above) other protocol buffer message types, allowing you to structure your data hierarchically. You can specify optional fields, required fields, and repeated fields. You can find more information about writing .proto files in the Protocol Buffer Language Guide.

Once you've defined your messages, you run the protocol buffer compiler for your application's language on your .proto file to generate data access classes. These provide simple accessors for each field (like name() and set_name()) as well as methods to serialize/parse the whole structure to/from raw bytes – so, for instance, if your chosen language is C++, running the compiler on the above example will generate a class called Person. You can then use this class in your application to populate, serialize, and retrieve Person protocol buffer messages. You might then write some code like this:

 
  1. Person person;
    person.set_name("John Doe");
    person.set_id(1234);
    person.set_email("jdoe@example.com");
    fstream output("myfile", ios::out | ios::binary);
    person.SerializeToOstream(&output);

Then, later on, you could read your message back in:

 
  1. fstream input("myfile", ios::in | ios::binary);
    Person person;
    person.ParseFromIstream(&input);
    cout << "Name: " << person.name() << endl;
    cout << "E-mail: " << person.email() << endl;

You can add new fields to your message formats without breaking backwards-compatibility; old binaries simply ignore the new field when parsing. So if you have a communications protocol that uses protocol buffers as its data format, you can extend your protocol without having to worry about breaking existing code.

You'll find a complete reference for using generated protocol buffer code in the API Reference section, and you can find out more about how protocol buffer messages are encoded in Protocol Buffer Encoding.

Why not just use XML?

Protocol buffers have many advantages over XML for serializing structured data. Protocol buffers:

  • are simpler
  • are 3 to 10 times smaller
  • are 20 to 100 times faster
  • are less ambiguous
  • generate data access classes that are easier to use programmatically

For example, let's say you want to model a person with a name and an email. In XML, you need to do:

 
  1. <person>
  2. <name>John Doe</name>
  3. <email>jdoe@example.com</email>
  4. </person>

while the corresponding protocol buffer message (in protocol buffer text format) is:

 
  1. # Textual representation of a protocol buffer.
  2. # This is *not* the binary format used on the wire.
  3. person {
  4. name: "John Doe"
  5. email: "jdoe@example.com"
  6. }

When this message is encoded to the protocol buffer binary format (the text format above is just a convenient human-readable representation for debugging and editing), it would probably be 28 bytes long and take around 100-200 nanoseconds to parse. The XML version is at least 69 bytes if you remove whitespace, and would take around 5,000-10,000 nanoseconds to parse.

Also, manipulating a protocol buffer is much easier:

 
  1.   cout << "Name: " << person.name() << endl;
      cout << "E-mail: " << person.email() << endl;

Whereas with XML you would have to do something like:

 
  1.   cout << "Name: "
           << person.getElementsByTagName("name")->item(0)->innerText()
           << endl;
      cout << "E-mail: "
           << person.getElementsByTagName("email")->item(0)->innerText()
           << endl;

However, protocol buffers are not always a better solution than XML – for instance, protocol buffers would not be a good way to model a text-based document with markup (e.g. HTML), since you cannot easily interleave structure with text. In addition, XML is human-readable and human-editable; protocol buffers, at least in their native format, are not. XML is also – to some extent – self-describing. A protocol buffer is only meaningful if you have the message definition (the .proto file).

编译器

binary format <--- the text format

https://github.com/protocolbuffers/protobuf/releases/tag/v3.7.1

Protocol Buffers source code is hosted on GitHub: https://github.com/protocolbuffers/protobuf.

Our old Google Code repository is: https://code.google.com/p/protobuf/. We moved to GitHub on Aug 26, 2014 and no future changes will be made on the Google Code site. For latest code updates/issues, please visit our GitHub site.

Compiling Your Protocol Buffers

https://developers.google.com/protocol-buffers/docs/pythontutorial

  1. protoc -I=$SRC_DIR --python_out=$DST_DIR $SRC_DIR/addressbook.proto

D:\pyPlusGlang\protoc-3.7.1-win64\bin>protoc -I=D:\pyPlusGlang\mysite\polls\ --python_out=D:\pyPlusG
lang\mysite\polls\ D:\pyPlusGlang\mysite\polls\addressbook.proto

D:\pyPlusGlang\mysite\polls\addressbook_pb2.py

  1. # -*- coding: utf-8 -*-
  2. # Generated by the protocol buffer compiler. DO NOT EDIT!
  3. # source: addressbook.proto
  4.  
  5. import sys
  6. _b=sys.version_info[0]<3 and (lambda x:x) or (lambda x:x.encode('latin1'))
  7. from google.protobuf import descriptor as _descriptor
  8. from google.protobuf import message as _message
  9. from google.protobuf import reflection as _reflection
  10. from google.protobuf import symbol_database as _symbol_database
  11. # @@protoc_insertion_point(imports)
  12.  
  13. _sym_db = _symbol_database.Default()
  14.  
  15. from google.protobuf import timestamp_pb2 as google_dot_protobuf_dot_timestamp__pb2
  16.  
  17. DESCRIPTOR = _descriptor.FileDescriptor(
  18. name='addressbook.proto',
  19. package='tutorial',
  20. syntax='proto3',
  21. serialized_options=_b('\n\024com.example.tutorialB\021AddressBookProtos\252\002$Google.Protobuf.Examples.AddressBook'),
  22. serialized_pb=_b('\n\x11\x61\x64\x64ressbook.proto\x12\x08tutorial\x1a\x1fgoogle/protobuf/timestamp.proto\"\x87\x02\n\x06Person\x12\x0c\n\x04name\x18\x01 \x01(\t\x12\n\n\x02id\x18\x02 \x01(\x05\x12\r\n\x05\x65mail\x18\x03 \x01(\t\x12,\n\x06phones\x18\x04 \x03(\x0b\x32\x1c.tutorial.Person.PhoneNumber\x12\x30\n\x0clast_updated\x18\x05 \x01(\x0b\x32\x1a.google.protobuf.Timestamp\x1aG\n\x0bPhoneNumber\x12\x0e\n\x06number\x18\x01 \x01(\t\x12(\n\x04type\x18\x02 \x01(\x0e\x32\x1a.tutorial.Person.PhoneType\"+\n\tPhoneType\x12\n\n\x06MOBILE\x10\x00\x12\x08\n\x04HOME\x10\x01\x12\x08\n\x04WORK\x10\x02\"/\n\x0b\x41\x64\x64ressBook\x12 \n\x06people\x18\x01 \x03(\x0b\x32\x10.tutorial.PersonBP\n\x14\x63om.example.tutorialB\x11\x41\x64\x64ressBookProtos\xaa\x02$Google.Protobuf.Examples.AddressBookb\x06proto3')
  23. ,
  24. dependencies=[google_dot_protobuf_dot_timestamp__pb2.DESCRIPTOR,])
  25.  
  26. _PERSON_PHONETYPE = _descriptor.EnumDescriptor(
  27. name='PhoneType',
  28. full_name='tutorial.Person.PhoneType',
  29. filename=None,
  30. file=DESCRIPTOR,
  31. values=[
  32. _descriptor.EnumValueDescriptor(
  33. name='MOBILE', index=0, number=0,
  34. serialized_options=None,
  35. type=None),
  36. _descriptor.EnumValueDescriptor(
  37. name='HOME', index=1, number=1,
  38. serialized_options=None,
  39. type=None),
  40. _descriptor.EnumValueDescriptor(
  41. name='WORK', index=2, number=2,
  42. serialized_options=None,
  43. type=None),
  44. ],
  45. containing_type=None,
  46. serialized_options=None,
  47. serialized_start=285,
  48. serialized_end=328,
  49. )
  50. _sym_db.RegisterEnumDescriptor(_PERSON_PHONETYPE)
  51.  
  52. _PERSON_PHONENUMBER = _descriptor.Descriptor(
  53. name='PhoneNumber',
  54. full_name='tutorial.Person.PhoneNumber',
  55. filename=None,
  56. file=DESCRIPTOR,
  57. containing_type=None,
  58. fields=[
  59. _descriptor.FieldDescriptor(
  60. name='number', full_name='tutorial.Person.PhoneNumber.number', index=0,
  61. number=1, type=9, cpp_type=9, label=1,
  62. has_default_value=False, default_value=_b("").decode('utf-8'),
  63. message_type=None, enum_type=None, containing_type=None,
  64. is_extension=False, extension_scope=None,
  65. serialized_options=None, file=DESCRIPTOR),
  66. _descriptor.FieldDescriptor(
  67. name='type', full_name='tutorial.Person.PhoneNumber.type', index=1,
  68. number=2, type=14, cpp_type=8, label=1,
  69. has_default_value=False, default_value=0,
  70. message_type=None, enum_type=None, containing_type=None,
  71. is_extension=False, extension_scope=None,
  72. serialized_options=None, file=DESCRIPTOR),
  73. ],
  74. extensions=[
  75. ],
  76. nested_types=[],
  77. enum_types=[
  78. ],
  79. serialized_options=None,
  80. is_extendable=False,
  81. syntax='proto3',
  82. extension_ranges=[],
  83. oneofs=[
  84. ],
  85. serialized_start=212,
  86. serialized_end=283,
  87. )
  88.  
  89. _PERSON = _descriptor.Descriptor(
  90. name='Person',
  91. full_name='tutorial.Person',
  92. filename=None,
  93. file=DESCRIPTOR,
  94. containing_type=None,
  95. fields=[
  96. _descriptor.FieldDescriptor(
  97. name='name', full_name='tutorial.Person.name', index=0,
  98. number=1, type=9, cpp_type=9, label=1,
  99. has_default_value=False, default_value=_b("").decode('utf-8'),
  100. message_type=None, enum_type=None, containing_type=None,
  101. is_extension=False, extension_scope=None,
  102. serialized_options=None, file=DESCRIPTOR),
  103. _descriptor.FieldDescriptor(
  104. name='id', full_name='tutorial.Person.id', index=1,
  105. number=2, type=5, cpp_type=1, label=1,
  106. has_default_value=False, default_value=0,
  107. message_type=None, enum_type=None, containing_type=None,
  108. is_extension=False, extension_scope=None,
  109. serialized_options=None, file=DESCRIPTOR),
  110. _descriptor.FieldDescriptor(
  111. name='email', full_name='tutorial.Person.email', index=2,
  112. number=3, type=9, cpp_type=9, label=1,
  113. has_default_value=False, default_value=_b("").decode('utf-8'),
  114. message_type=None, enum_type=None, containing_type=None,
  115. is_extension=False, extension_scope=None,
  116. serialized_options=None, file=DESCRIPTOR),
  117. _descriptor.FieldDescriptor(
  118. name='phones', full_name='tutorial.Person.phones', index=3,
  119. number=4, type=11, cpp_type=10, label=3,
  120. has_default_value=False, default_value=[],
  121. message_type=None, enum_type=None, containing_type=None,
  122. is_extension=False, extension_scope=None,
  123. serialized_options=None, file=DESCRIPTOR),
  124. _descriptor.FieldDescriptor(
  125. name='last_updated', full_name='tutorial.Person.last_updated', index=4,
  126. number=5, type=11, cpp_type=10, label=1,
  127. has_default_value=False, default_value=None,
  128. message_type=None, enum_type=None, containing_type=None,
  129. is_extension=False, extension_scope=None,
  130. serialized_options=None, file=DESCRIPTOR),
  131. ],
  132. extensions=[
  133. ],
  134. nested_types=[_PERSON_PHONENUMBER, ],
  135. enum_types=[
  136. _PERSON_PHONETYPE,
  137. ],
  138. serialized_options=None,
  139. is_extendable=False,
  140. syntax='proto3',
  141. extension_ranges=[],
  142. oneofs=[
  143. ],
  144. serialized_start=65,
  145. serialized_end=328,
  146. )
  147.  
  148. _ADDRESSBOOK = _descriptor.Descriptor(
  149. name='AddressBook',
  150. full_name='tutorial.AddressBook',
  151. filename=None,
  152. file=DESCRIPTOR,
  153. containing_type=None,
  154. fields=[
  155. _descriptor.FieldDescriptor(
  156. name='people', full_name='tutorial.AddressBook.people', index=0,
  157. number=1, type=11, cpp_type=10, label=3,
  158. has_default_value=False, default_value=[],
  159. message_type=None, enum_type=None, containing_type=None,
  160. is_extension=False, extension_scope=None,
  161. serialized_options=None, file=DESCRIPTOR),
  162. ],
  163. extensions=[
  164. ],
  165. nested_types=[],
  166. enum_types=[
  167. ],
  168. serialized_options=None,
  169. is_extendable=False,
  170. syntax='proto3',
  171. extension_ranges=[],
  172. oneofs=[
  173. ],
  174. serialized_start=330,
  175. serialized_end=377,
  176. )
  177.  
  178. _PERSON_PHONENUMBER.fields_by_name['type'].enum_type = _PERSON_PHONETYPE
  179. _PERSON_PHONENUMBER.containing_type = _PERSON
  180. _PERSON.fields_by_name['phones'].message_type = _PERSON_PHONENUMBER
  181. _PERSON.fields_by_name['last_updated'].message_type = google_dot_protobuf_dot_timestamp__pb2._TIMESTAMP
  182. _PERSON_PHONETYPE.containing_type = _PERSON
  183. _ADDRESSBOOK.fields_by_name['people'].message_type = _PERSON
  184. DESCRIPTOR.message_types_by_name['Person'] = _PERSON
  185. DESCRIPTOR.message_types_by_name['AddressBook'] = _ADDRESSBOOK
  186. _sym_db.RegisterFileDescriptor(DESCRIPTOR)
  187.  
  188. Person = _reflection.GeneratedProtocolMessageType('Person', (_message.Message,), dict(
  189.  
  190. PhoneNumber = _reflection.GeneratedProtocolMessageType('PhoneNumber', (_message.Message,), dict(
  191. DESCRIPTOR = _PERSON_PHONENUMBER,
  192. __module__ = 'addressbook_pb2'
  193. # @@protoc_insertion_point(class_scope:tutorial.Person.PhoneNumber)
  194. ))
  195. ,
  196. DESCRIPTOR = _PERSON,
  197. __module__ = 'addressbook_pb2'
  198. # @@protoc_insertion_point(class_scope:tutorial.Person)
  199. ))
  200. _sym_db.RegisterMessage(Person)
  201. _sym_db.RegisterMessage(Person.PhoneNumber)
  202.  
  203. AddressBook = _reflection.GeneratedProtocolMessageType('AddressBook', (_message.Message,), dict(
  204. DESCRIPTOR = _ADDRESSBOOK,
  205. __module__ = 'addressbook_pb2'
  206. # @@protoc_insertion_point(class_scope:tutorial.AddressBook)
  207. ))
  208. _sym_db.RegisterMessage(AddressBook)
  209.  
  210. DESCRIPTOR._options = None
  211. # @@protoc_insertion_point(module_scope)

https://developers.google.com/protocol-buffers/docs/pythontutorial

protobuf2  protobuf3 差异

addressbook.proto: Explicit default values are not allowed in proto3.
addressbook.proto: Required fields are not allowed in proto3.

addressbook.proto:8:12: Explicit 'optional' labels are disallowed in the Proto3 syntax. To define 'o
ptional' fields in Proto3, simply remove the 'optional' label, as fields are 'optional' by default.

Language Guide  |  Protocol Buffers  |  Google Developers
https://developers.google.com/protocol-buffers/docs/proto

读写消息 protobuf二进制文件

  1. D:\pyPlusGlang\LProtocol-buffers>python pb2ReadingAMessage.py myADDRESS_BOOK_FILE
  2. Person ID: 1
  3. Name: a001
  4. Can't test non-submessage field "Person.email" for presence in proto3.
  5. Mobile phone #:
  6. 152001
  7.  
  8. D:\pyPlusGlang\LProtocol-buffers>python pb2WritingAMessage.py myADDRESS_BOOK_FILE
  9. ['pb2WritingAMessage.py', 'myADDRESS_BOOK_FILE']
  10. Enter person ID number: 002
  11. Enter name: a002
  12. Enter email address (blank for none): b@qq.cn
  13. Enter a phone number (or leave blank to finish): 142002
  14. Is this a mobile, home, or work phone? home
  15. Enter a phone number (or leave blank to finish):
  16.  
  17. D:\pyPlusGlang\LProtocol-buffers>python pb2ReadingAMessage.py myADDRESS_BOOK_FILE
  18. Person ID: 1
  19. Name: a001
  20. Can't test non-submessage field "Person.email" for presence in proto3.
  21. Mobile phone #:
  22. 152001
  23. Person ID: 2
  24. Name: a002
  25. Can't test non-submessage field "Person.email" for presence in proto3.
  26. Home phone #:
  27. 142002
  28.  
  29. D:\pyPlusGlang\LProtocol-buffers>

address_book = addressbook_pb2.AddressBook()
写protobuf文件
# Add an address.
PromptForAddress(address_book.people.add())
# Write the new address book back to disk.
f = open(fname, "wb")
f.write(address_book.SerializeToString())

读protobuf文件
# Read the existing address book.
f = open(fname, "rb")
address_book.ParseFromString(f.read())

一阶

python生成的protobuf文件被golang读取 反序列化

生成golang

golang/protobuf: Go support for Google's protocol buffers
https://github.com/golang/protobuf

  • Grab the code from the repository and install the proto package. The simplest way is to run go get -u github.com/golang/protobuf/protoc-gen-go. The compiler plugin, protoc-gen-go, will be installed in $GOPATH/bin unless $GOBIN is set. It must be in your $PATH for the protocol compiler, protoc, to find it.

protoc -I=$SRC_DIR --go_out=$DST_DIR $SRC_DIR/addressbook.proto

protoc -I=D:\pyPlusGlang\LProtocol-buffers --go_out=D:\pyPlusGlang\LProtocol-buffers D:\pyPlusGlang\LProtocol-buffers\SearchRequest.proto

protoc.exe 需要protoc-gen-go.exe,会从当前目录开始查找

python

成功读写文件

  1. D:\pyPlusGlang\LProtocol-buffers>python pb3ReadingAMessage.py sRqs_pb3
  2. query: appleVSbanana
  3. page_number: 1
  4. result_per_page: 4
  5.  
  6. D:\pyPlusGlang\LProtocol-buffers>python pb3WritingAMessage.py sRqs_pb3
  7. ['pb3WritingAMessage.py', 'sRqs_pb3']
  8. Enter query: aANDb
  9. Enter page_number: 123
  10. Enter result_per_page: 789
  11.  
  12. D:\pyPlusGlang\LProtocol-buffers>python pb3ReadingAMessage.py sRqs_pb3
  13. query: appleVSbanana
  14. page_number: 1
  15. result_per_page: 4
  16. query: aANDb
  17. page_number: 123
  18. result_per_page: 789
  19.  
  20. D:\pyPlusGlang\LProtocol-buffers>

  

  1. syntax = "proto3";
  2.  
  3. message SearchRequest {
  4. string query = 1;
  5. int32 page_number = 2;
  6. int32 result_per_page = 3;
  7. }
  8.  
  9. message SearchRequestList {
  10. repeated SearchRequest sR = 1;
  11. }

  D:\pyPlusGlang\LProtocol-buffers\SearchRequestList_pb2.py

  1. # -*- coding: utf-8 -*-
  2. # Generated by the protocol buffer compiler. DO NOT EDIT!
  3. # source: SearchRequestList.proto
  4.  
  5. import sys
  6. _b=sys.version_info[0]<3 and (lambda x:x) or (lambda x:x.encode('latin1'))
  7. from google.protobuf import descriptor as _descriptor
  8. from google.protobuf import message as _message
  9. from google.protobuf import reflection as _reflection
  10. from google.protobuf import symbol_database as _symbol_database
  11. # @@protoc_insertion_point(imports)
  12.  
  13. _sym_db = _symbol_database.Default()
  14.  
  15. DESCRIPTOR = _descriptor.FileDescriptor(
  16. name='SearchRequestList.proto',
  17. package='',
  18. syntax='proto3',
  19. serialized_options=None,
  20. serialized_pb=_b('\n\x17SearchRequestList.proto\"L\n\rSearchRequest\x12\r\n\x05query\x18\x01 \x01(\t\x12\x13\n\x0bpage_number\x18\x02 \x01(\x05\x12\x17\n\x0fresult_per_page\x18\x03 \x01(\x05\"/\n\x11SearchRequestList\x12\x1a\n\x02sR\x18\x01 \x03(\x0b\x32\x0e.SearchRequestb\x06proto3')
  21. )
  22.  
  23. _SEARCHREQUEST = _descriptor.Descriptor(
  24. name='SearchRequest',
  25. full_name='SearchRequest',
  26. filename=None,
  27. file=DESCRIPTOR,
  28. containing_type=None,
  29. fields=[
  30. _descriptor.FieldDescriptor(
  31. name='query', full_name='SearchRequest.query', index=0,
  32. number=1, type=9, cpp_type=9, label=1,
  33. has_default_value=False, default_value=_b("").decode('utf-8'),
  34. message_type=None, enum_type=None, containing_type=None,
  35. is_extension=False, extension_scope=None,
  36. serialized_options=None, file=DESCRIPTOR),
  37. _descriptor.FieldDescriptor(
  38. name='page_number', full_name='SearchRequest.page_number', index=1,
  39. number=2, type=5, cpp_type=1, label=1,
  40. has_default_value=False, default_value=0,
  41. message_type=None, enum_type=None, containing_type=None,
  42. is_extension=False, extension_scope=None,
  43. serialized_options=None, file=DESCRIPTOR),
  44. _descriptor.FieldDescriptor(
  45. name='result_per_page', full_name='SearchRequest.result_per_page', index=2,
  46. number=3, type=5, cpp_type=1, label=1,
  47. has_default_value=False, default_value=0,
  48. message_type=None, enum_type=None, containing_type=None,
  49. is_extension=False, extension_scope=None,
  50. serialized_options=None, file=DESCRIPTOR),
  51. ],
  52. extensions=[
  53. ],
  54. nested_types=[],
  55. enum_types=[
  56. ],
  57. serialized_options=None,
  58. is_extendable=False,
  59. syntax='proto3',
  60. extension_ranges=[],
  61. oneofs=[
  62. ],
  63. serialized_start=27,
  64. serialized_end=103,
  65. )
  66.  
  67. _SEARCHREQUESTLIST = _descriptor.Descriptor(
  68. name='SearchRequestList',
  69. full_name='SearchRequestList',
  70. filename=None,
  71. file=DESCRIPTOR,
  72. containing_type=None,
  73. fields=[
  74. _descriptor.FieldDescriptor(
  75. name='sR', full_name='SearchRequestList.sR', index=0,
  76. number=1, type=11, cpp_type=10, label=3,
  77. has_default_value=False, default_value=[],
  78. message_type=None, enum_type=None, containing_type=None,
  79. is_extension=False, extension_scope=None,
  80. serialized_options=None, file=DESCRIPTOR),
  81. ],
  82. extensions=[
  83. ],
  84. nested_types=[],
  85. enum_types=[
  86. ],
  87. serialized_options=None,
  88. is_extendable=False,
  89. syntax='proto3',
  90. extension_ranges=[],
  91. oneofs=[
  92. ],
  93. serialized_start=105,
  94. serialized_end=152,
  95. )
  96.  
  97. _SEARCHREQUESTLIST.fields_by_name['sR'].message_type = _SEARCHREQUEST
  98. DESCRIPTOR.message_types_by_name['SearchRequest'] = _SEARCHREQUEST
  99. DESCRIPTOR.message_types_by_name['SearchRequestList'] = _SEARCHREQUESTLIST
  100. _sym_db.RegisterFileDescriptor(DESCRIPTOR)
  101.  
  102. SearchRequest = _reflection.GeneratedProtocolMessageType('SearchRequest', (_message.Message,), dict(
  103. DESCRIPTOR = _SEARCHREQUEST,
  104. __module__ = 'SearchRequestList_pb2'
  105. # @@protoc_insertion_point(class_scope:SearchRequest)
  106. ))
  107. _sym_db.RegisterMessage(SearchRequest)
  108.  
  109. SearchRequestList = _reflection.GeneratedProtocolMessageType('SearchRequestList', (_message.Message,), dict(
  110. DESCRIPTOR = _SEARCHREQUESTLIST,
  111. __module__ = 'SearchRequestList_pb2'
  112. # @@protoc_insertion_point(class_scope:SearchRequestList)
  113. ))
  114. _sym_db.RegisterMessage(SearchRequestList)
  115.  
  116. # @@protoc_insertion_point(module_scope)

  

D:\pyPlusGlang\LProtocol-buffers\pb3WritingAMessage.py

  1. #! /usr/bin/python
  2. import SearchRequestList_pb2
  3. import sys
  4.  
  5. def PromptForSearchRequest(searchRequest):
  6. searchRequest.query = input("Enter query: ")
  7. searchRequest.page_number = int(input("Enter page_number: "))
  8. searchRequest.result_per_page = int(input("Enter result_per_page: "))
  9. print(sys.argv)
  10. if len(sys.argv) != 2:
  11. print("here is :", __file__, sys._getframe().f_lineno) # @print@ "Usage:", sys.argv[0], "SearchRequest__FILE"
  12. sys.exit(-1)
  13.  
  14. searchRequestList = SearchRequestList_pb2.SearchRequestList()
  15.  
  16. # Read the existing SearchRequest book.
  17. try:
  18. f = open(sys.argv[1], "rb")
  19. searchRequestList.ParseFromString(f.read())
  20. f.close()
  21. except IOError:
  22. print("here is :", __file__,
  23. sys._getframe().f_lineno) # @print@ sys.argv[1] + ": Could not open file. Creating a new one."
  24.  
  25. # Add an SearchRequest.
  26. PromptForSearchRequest(searchRequestList.sR.add())
  27.  
  28. # Write the new SearchRequest book back to disk.
  29. f = open(sys.argv[1], "wb")
  30. f.write(searchRequestList.SerializeToString())
  31. f.close()

D:\pyPlusGlang\LProtocol-buffers\pb3ReadingAMessage.py

  1. #! /usr/bin/python
  2.  
  3. import SearchRequestList_pb2
  4. import sys
  5.  
  6. # Iterates though all people in the AddressBook and prints info about them.
  7. def ListSr(searchRequestList):
  8. for sR in searchRequestList.sR:
  9. print("query:", sR.query)
  10. print("page_number:", sR.page_number)
  11. print("result_per_page:", sR.result_per_page)
  12.  
  13. # Main procedure: Reads the entire address book from a file and prints all
  14. # the information inside.
  15. if len(sys.argv) != 2:
  16. print("Usage:", sys.argv[0], "searchRequestList")
  17. sys.exit(-1)
  18.  
  19. searchRequestList = SearchRequestList_pb2.SearchRequestList()
  20.  
  21. # Read the existing address book.
  22. f = open(sys.argv[1], "rb")
  23. searchRequestList.ParseFromString(f.read())
  24. f.close()
  25.  
  26. ListSr(searchRequestList)

  

  

protobuf protocol-buffers 序列化数据 gobs pickling string XML 用C实现的cPickle比pickle快1000倍 protobuf2 protobuf3 差异的更多相关文章

  1. protobuf Protocol Buffers 简介 案例 MD

    Markdown版本笔记 我的GitHub首页 我的博客 我的微信 我的邮箱 MyAndroidBlogs baiqiantao baiqiantao bqt20094 baiqiantao@sina ...

  2. Xml,Json,Hessian,Protocol Buffers序列化对比

    简介 这篇博客主要对Xml,Json,Hessian,Protocol Buffers的序列化和反序列化性能进行对比,Xml和Json的基本概念就不说了. Hessian:Hessian是一个轻量级的 ...

  3. protobuf(Protocol Buffers)java初体验

    因为项目须要所以简单的研究了下protobuf.我也是參照网上的博客,所以大部分内容我也就不反复造轮子了.首先protobuf介绍点击这里,使用介绍点击这里,使用demo看这里. 我个人的第一个样例也 ...

  4. Protocol Buffers序列化原理

    1. 使用Varint编码数据,越小的数据,用少的字节编码.如小于128的用一个字节编码,大于128的用多个字节编码.同时,每个字节最高为1或者0表示是否为数字的一部分. 2. 由于负数的补码表示很大 ...

  5. Hadoop基础-Protocol Buffers串行化与反串行化

    Hadoop基础-Protocol Buffers串行化与反串行化 作者:尹正杰 版权声明:原创作品,谢绝转载!否则将追究法律责任. 我们之前学习过很多种序列化文件格式,比如python中的pickl ...

  6. ProtoBuf3语法指南(Protocol Buffers)_上

    0.说明 ProtoBuf3语法指南, 又称为proto3, 是谷歌的Protocol Buffers第3个版本. 本文基于官方英文版本翻译, 加上了自己的理解少量修改, 一共分为上下两部分. 1.序 ...

  7. Netty中如何序列化数据

    JDK提供了ObjectOutputStream和ObjectInputStream,用于通过网络对POJO的基本数据类型和图进行序列化和反序列化.该API并不复杂,而且可以被应用于任何实现了java ...

  8. Netty(五):Netty中如何序列化数据

    JDK提供了ObjectOutputStream和ObjectInputStream,用于通过网络对POJO的基本数据类型和图进行序列化和反序列化.该API并不复杂,而且可以被应用于任何实现了java ...

  9. 使用纳米 Protocol buffers 作为序列化数据

    使用纳米 Protocol buffers 作为序列化数据 Protocol Buffers 是 Google 公司开发的一种数据描述语言,类似于XML能够将结构化数据序列化. 但是它更小, 更快, ...

随机推荐

  1. Linux之virtualenv和virtualenvwrapper

    一,介绍 在使用 Python 开发的过程中,工程一多,难免会碰到不同的工程依赖不同版本的库的问题:亦或者是在开发过程中不想让物理环境里充斥各种各样的库,引发未来的依赖灾难.此时,我们需要对于不同的工 ...

  2. 动态URL是什么?动态URL有什么特点?

    动态URL是什么动态URL就是动态页面,动态链接,即指在URL中出现“?”这样的参数符号,并以aspx.asp.jsp.php.perl.cgi为后缀的url. 动态URL有什么特点1.在建设反向链接 ...

  3. 【转载】Redis 4.0 自动内存碎片整理(Active Defrag)源码分析

    click原文链接原文链接:https://blog.csdn.net/zouhuajianclever/article/details/90669409阅读本文前建议先阅读此篇博客: Redis源码 ...

  4. 基于beautifulSoup进行电影网站排名的获取与格式化输出

    要求 编写代码完成以下任务: ① 将地址"http://www.cbooo.cn/year?year=2019"源代码使用任意方法保存到指定文件中(文件类型不限). ② 使用文件流 ...

  5. MySQL分布式数据库架构:分库、分表、排序、分页、分组、实现教程

    MySQL分库分表总结: 单库单表 : 单库单表是最常见的数据库设计,例如,有一张用户(user)表放在数据库db中,所有的用户都可以在db库中的user表中查到. 单库多表 : 随着用户数量的增加, ...

  6. MySQL数据库语法-单表查询练习

    MySQL数据库语法-单表查询练习 作者:尹正杰 版权声明:原创作品,谢绝转载!否则将追究法律责任. 本篇博客主要是对聚合函数和分组的练习. 一.数据表和测试数据准备 /* @author :yinz ...

  7. Scala环境安装步骤

    1.scala解释器本地安装 2.IDEA安装 3.安装IDEA的scala插件 4.创建maven项目 5.安装ScalaSDK

  8. Luogu P1290 欧几里得的游戏/UVA10368 Euclid's Game

    Luogu P1290 欧几里得的游戏/UVA10368 Euclid's Game 对于博弈论的题目没接触过多少,而这道又是比较经典的SG博弈,所以就只能自己来推关系-- 假设我们有两个数$m,n$ ...

  9. FontLab

    FontLab 字体制作软件

  10. javaWeb上传

    上传(上传不能使用BaseServlet) 1. 上传对表单限制  * method="post"  * enctype="multipart/form-data&quo ...