序列化－pickle，json
在惠州等地区，都构建了全面的区域性战略布局，加强发展的系统性、市场前瞻性、产品创新能力，以专注、极致的服务理念，为客户提供网站建设、
成都网站设计 网站设计制作按需设计,公司网站建设,企业网站建设,
成都品牌网站建设,
全网整合营销推广,
成都外贸网站建设,惠州网站建设费用合理。
1、序列化
将对象的状态信息转换为可以存储或者可以传输的形式过程
内存中有一些结构数据，希望保存下来冲用，或者发送给别人使用
很多游戏允许你在退出的时候保存进度，然后你再次启动的时候回到退出的地方。
2、常用的一些序列化
pickle，cPickle
JSON
Shelve
YAML
3、序列化对象到磁盘，所有的python支持的类型都可以用pickle做序列化
序列化到磁盘：pickle.dump(obj, file)，从磁盘反序列化：pickle.load(file)
序列化到内存：pickle.dumps(obj)，从内存反序列化：pickle.loads(str)
＃pickle序列化到磁盘,pickle.dump(参数1，参数2)，fd是问价句炳
In [1]: import pickle
In [2]: entry = {'a':11, 'b':22}
In [3]: with open('/tmp/1.pickle','wb') as fd: #w表示写方式，b表示二进制
...: pickle.dump(entry, fd)
[root@133 ~]# file /tmp/1.pickle
/tmp/1.pickle: ASCII text
[root@133 ~]cat /tmp/1.pickle
(dp0
S'a'
p1
I11
sS'b'
p2
I22
#pickle.load(fd)从硬盘反序列化，
In [7]: with open('/tmp/1.pickle') as fd:
...: a = pickle.load(fd)
...:
In [8]: a
Out[8]: {'a': 11, 'b': 22}
#pickle序列化数据到内存中，
In [14]: entry = {'a':11, 'b':22}
In [15]: b = pickle.dumps(entry)
In [16]: b
Out[16]: "(dp0\nS'a'\np1\nI11\nsS'b'\np2\nI22\ns."
#pickle.loads(fd)从内存反序列化
In [17]: entry1=pickle.loads(b)
In [19]: entry1
Out[19]: {'a': 11, 'b': 22}
pickle简单应用
[root@133 tmp]# cd /opt/python/django/
[root@133 django]# cp -pr simplecmdb simplecmdbbak
[root@133 simplecmdb]# cd /opt/python/django/simplecmdbbak/simplecmdb
[root@133 simplecmdb]# vim settings.py
#注释掉默认的sqlite3数据库
#DATABASES = {
# 'default': {
# 'ENGINE': 'django.db.backends.sqlite3',
# 'NAME': os.path.join(BASE_DIR, 'db.sqlite3'),
# }
#}
#指定使用mysql数据库
DATABASES = {
'default':{
'ENGINE':'django.db.backends.mysql',
'NAME':'cmdb',
'USER':'root',
'PASSWORD':'Amos!@#$',
'HOST':'127.0.0.1',
'PORT':'3306',
}
}
登录mysql，创建数据库cmdb，在数据库cmdb中创建数据表
[root@133 simplecmdbbak]# cd /opt/python/django/simplecmdbbak
[root@133 simplecmdbbak]# rm -rf db.sqlite3 #删除sqlite3的数据库
[root@133 simplecmdbbak]# cd hostinfo/
[root@133 hostinfo]# vim models.py #models.py已经定义了数据表，只要同步即可在mysql中建表
from django.db import models
# Create your models here.
class Host(models.Model):
hostname = models.CharField(max_length = 50)
ip = models.IPAddressField()
vendor = models.CharField(max_length = 50)
product = models.CharField(max_length = 50)
sn = models.CharField(max_length = 50)
cpu_model = models.CharField(max_length = 50)
cpu_num = models.IntegerField(max_length = 50)
memory = models.CharField(max_length = 50)
osver = models.CharField(max_length = 50)
#查看数据库配置文件有没有错误
[root@133 simplecmdbbak]# python manage.py validate
0 errors found
mysql> create database cmdb;
Query OK, 1 row affected (0.00 sec)
[root@133 simplecmdbbak]# python manage.py syncdb
Creating tables ...
Creating table django_admin_log
Creating table auth_permission
Creating table auth_group_permissions
Creating table auth_group
Creating table auth_user_groups
Creating table auth_user_user_permissions
Creating table auth_user
Creating table django_content_type
Creating table django_session
Creating table hostinfo_host
Creating table hostinfo_hostgroup_members
Creating table hostinfo_hostgroup
You just installed Django's auth system, which means you don't have any superusers defined.
Would you like to create one now? (yes/no): yes
Username (leave blank to use 'root'): root
Email address: 1350368559@qq.com
Password:
Password (again):
Superuser created successfully.
Installing custom SQL ...
Installing indexes ...
Installed 0 object(s) from 0 fixture(s)
[root@133 simplecmdbbak]# mysql -uroot -p
Enter password:
mysql> use cmdb;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A
Database changed
mysql> show tables;
+----------------------------+
| Tables_in_cmdb |
+----------------------------+
| auth_group |
| auth_group_permissions |
| auth_permission |
| auth_user |
| auth_user_groups |
| auth_user_user_permissions |
| django_admin_log |
| django_content_type |
| django_session |
| hostinfo_host |
| hostinfo_hostgroup |
| hostinfo_hostgroup_members |
+----------------------------+
12 rows in set (0.00 sec)
mysql> exit
Bye
修改系统收集信息
[root@133 django]# vim /opt/python/django/sysinformation-pickle.py
import pickle
# d = urllib.urlencode(dic)
d = pickle.dumps(dic) #使用pickle将收集到的信息保存到内存中
#修改视图文件，得到属性文件
[root@133 django]# vim /opt/python/django/simplecmdbbak/hostinfo/views.py
print req.body
启动django服务器
[root@133 simplecmdbbak]# python manage.py runserver 112.65.140.133:8080
执行系统收集脚本：python sysinformation-pickle.py，报错500，排查过程如下：
[root@133 django]# python sysinformation-pickle.py #由于后边的变量值没有收到具体的信息报错
Traceback (most recent call last):
File "sysinformation-pickle.py", line 118, in
req = urllib2.urlopen('http://112.65.140.133:8080/hostinfo/collect/',d)
File "/opt/amos/python2.7/lib/python2.7/urllib2.py", line 126, in urlopen
return _opener.open(url, data, timeout)
File "/opt/amos/python2.7/lib/python2.7/urllib2.py", line 406, in open
response = meth(req, response)
File "/opt/amos/python2.7/lib/python2.7/urllib2.py", line 519, in http_response
'http', request, response, code, msg, hdrs)
File "/opt/amos/python2.7/lib/python2.7/urllib2.py", line 444, in error
return self._call_chain(*args)
File "/opt/amos/python2.7/lib/python2.7/urllib2.py", line 378, in _call_chain
result = func(*args)
File "/opt/amos/python2.7/lib/python2.7/urllib2.py", line 527, in http_error_default
raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
urllib2.HTTPError: HTTP Error 500: INTERNAL SERVER ERROR
[root@133 simplecmdbbak]# python manage.py runserver 112.65.140.133:8080
Validating models...
0 errors found
January 15, 2017 - 18:00:56
Django version 1.6.5, using settings 'simplecmdb.settings'
Starting development server at http://112.65.140.133:8080/
Quit the server with CONTROL-C.
#以下是pickle发送到服务器的信息，格式是pickle格式，由simplecmdbbak/hostinfo/views.py处理得到的
(dp0
S'osver'
p1
S'CentOS release 6.7 (Final)'
p2
sS'product'
p3
S'PowerEdge R710'
p4
sS'vendor'
p5
S'Dell Inc.'
p6
sS'sn'
p7
S'4HBDT2X'
p8
sS'memory'
p9
S'19976M'
p10
sS'cpu_num'
p11
I8
sS'ip'
p12
(S'br1'
p13
S'112.65.140.133'
p14
S'A4:BA:DB:20:93:23'
p15
tp16
sS'hostname'
p17
S'133'
p18
sS'cpu_model'
p19
S'Intel(R) 2.00GHz'
p20
s.
None
None
None
None
None
None
None
None
None
[15/Jan/2017 18:01:04] "POST /hostinfo/collect/ HTTP/1.1" 500 118260
#测试，如果将：sysinformation-pickle.py修改为：
d = urllib.urlencode(dic)
# d = pickle.dumps(dic)
结果就是：
osver=CentOS+release+6.7+%28Final%29&product=PowerEdge+R710&vendor=Dell+Inc.&sn=4HBDT2X&memory=19976M&cpu_num=8&ip=%28%27br1%27%2C+%27112.65.140.133%27%2C+%27A4%3ABA%3ADB%3A20%3A93%3A23%27%29&hostname=133&cpu_model=Intel%28R%29+2.00GHz
133
('br1', '112.65.140.133', 'A4:BA:DB:20:93:23')
CentOS release 6.7 (Final)
Dell Inc.
PowerEdge R710
Intel(R) 2.00GHz
8
19976M
4HBDT2X
数据已经发送到djago这里，如何将数据发序列化出来成为字典，即可正常使用。
[root@133 django]# vim /opt/python/django/simplecmdbbak/hostinfo/views.py
from django.shortcuts import render
from django.http import HttpResponse
from hostinfo.models import Host
import pickle
# Create your views here.
def collect(req):
if req.POST:
print pickle.loads(req.body)
[root@133 django]# python sysinformation-pickle.py#结果显示已经去得了pickle loads出来的字典了
{'product': 'PowerEdge R710', 'vendor': 'Dell Inc.', 'cpu_num': 8, 'ip': ('br1', '112.65.140.133', 'A4:BA:DB:20:93:23'), 'hostname': '133', 'cpu_model': 'Intel(R) 2.00GHz', 'osver': 'CentOS release 6.7 (Final)', 'sn': '4HBDT2X', 'memory': '19976M'}
[root@133 django]# vim /opt/python/django/simplecmdbbak/hostinfo/views.py
import pickle
def collect(req):
if req.POST:
obj = pickle.loads(req.body)
hostname = obj['hostname']
print hostname
ip = obj['ip']
print ip
osver = obj['osver']
print osver
vendor = obj['vendor']
print vendor
product = obj['product']
print product
cpu_model = obj['cpu_model']
print cpu_model
cpu_num = obj['cpu_num']
print cpu_num
memory = obj['memory']
print memory
sn = obj['sn']
print sn
:10,27s/req.POST.get/obj/
:10,27s/(/[/
:10,27s/)/]/
[root@133 simplecmdb]# vim hostinfo/models.py
from django.db import models
# Create your models here.
class Host(models.Model):
hostname = models.CharField(max_length = 200)
ip = models.IPAddressField()
vendor = models.CharField(max_length = 200)
product = models.CharField(max_length = 200)
sn = models.CharField(max_length = 200)
cpu_model = models.CharField(max_length = 200)
cpu_num = models.IntegerField(max_length = 200)
memory = models.CharField(max_length = 200)
osver = models.CharField(max_length = 200)
def __unicode__(self):
return self.hostname
数据库字段的长度改为200，然后数据库cmdb删除，重新同步的，结果还是一样。500
搞不清为什么会这样？
另外一个问题：
#d = urllib.urlencode(dic)
d = pickle.dumps(dic)
req = urllib2.urlopen('http://112.65.140.133:8080/hostinfo/collect/',d)
print req.read() #d = urllib.urlencode(dic)
这里的d难道不用urlencode吗？不转换成urlencode格式，怎么能用urllib2.urlopen发送处理呢？
host是实例，不是字典，所以不能这样host['hostname']，只能host.hostname这样来调用属性。
可以在实例化（ host = Host()）之前来打印hostname、ip等信息，来进行拍错。
这里的d是序列化后的对象，这里可以使用pickle序列化，跟使用json一样。
视频里有三种方式
1. 使用urlencode
2. 使用pickle
3. 使用json
我的问题找到了，原因是：ip = models.IPAddressField() 无法保存元祖。
def collect(req):
if req.POST:
print pickle.loads(req.body)
obj = pickle.loads(req.body)
hostname = obj['hostname']
ip = obj['ip']
osver = obj['osver']
vendor = obj['vendor']
product = obj['product']
cpu_model = obj['cpu_model']
cpu_num = obj['cpu_num']
memory = obj['memory']
sn = obj['sn']
print sn
print 'pickle load ok'
host = Host()
host.hostname = hostname
host.ip = ip
host.osver = osver
host.vendor = vendor
host.product = product
host.cpu_model = cpu_model
host.cpu_num = cpu_num
host.memory = memory
host.sn = sn
print host.sn
host.save()
print "host save ok" ＃没有打印
print host.sn ＃没有打印
return HttpResponse('OK')
else:
return HttpResponse('no data')
结果是：
[16/Jan/2017 11:34:20] "POST /hostinfo/collect/ HTTP/1.1" 500 72226
{'product': 'PowerEdge R710', 'vendor': 'Dell Inc.', 'cpu_num': 8, 'ip': ('br1', '112.65.140.133', 'A4:BA:DB:20:93:23'), 'hostname': '133', 'cpu_model': 'Intel(R) 2.00GHz', 'osver': 'CentOS release 6.7 (Final)', 'sn': '4HBDT2X', 'memory': '19976M'}
4HBDT2X
pickle load ok
4HBDT2X
很明显是host.save()没有成功，但是我却不知道为什么没有成功？
最后发现是：
我打印出来的IP被我设置为三个参数：
def parseIfconfig(parsed_data):
dic = {}
tuple_addr= ('lo','vir','vnet','em3','em4')
parsed_data = [i for i in parsed_data if i and not i.startswith(tuple_addr)]
for lines in parsed_data:
line_list = lines.split('\n')
devname = line_list[0].split()[0]
macaddr = line_list[0].split()[-1]
ipaddr = line_list[1].split()[1].split(':')[1]
break
dic['ip'] = devname,ipaddr,macaddr
return dic
ip如果是：'ip': '112.65.140.133' 就可以保存了
而实际结果是：'ip': ('br1', '112.65.140.133', 'A4:BA:DB:20:93:23')
所以无法保存到数据库中，所以会报错
改为：
dic['ip'] = ipaddr
就ok了另外有需要云服务器可以了解下创新互联cdcxhl.cn，海内外云服务器15元起步，三天无理由+7*72小时售后在线，公司持有idc许可证，提供“云服务器、裸金属服务器、高防服务器、香港服务器、美国服务器、虚拟主机、免备案服务器”等云主机租用服务以及企业上云的综合解决方案，具有“安全稳定、简单易用、服务可用性高、性价比高”等特点与优势，专为企业上云打造定制，能够满足用户丰富、多元化的应用场景需求。
文章名称：序列化－pickle-创新互联
分享链接：http://cdxtjz.cn/article/dhcgoc.html
