Day28 Lab 2 - Object storage前端实作

我们的前端并不是一个精美的UI，在本次Lab中实做了前端有两个目的

1. 方便测试
2. 好的前端能减经後端的负担，例如上传前对资料进行压缩

那麽接下来让我们看看我们前端有什麽功能

• Upload Object

python frontend.py --type uo --name obj01 --version 1.7.3 --file /home/distributed\_obj\_storage\_system/distributed\_obj\_system/test

首先，在执行frontend.py，我们必须传递一个type parameter，指定要做什麽request，type的mapping如下所示

funcMap = {
      'uo': uploadObject,
      'vl': getVersionList,
      'do': downloadObject,
      'ol': getObjLocation,
}

https://github.com/kaichiachen/python-simple-object-storage/blob/master/distributed_obj_system/frontend.py#L114

uo mapping到uploadObject，uploadObject的程序码如下图所示

def uploadObject(args):
   content = args.content
   name = args.name
   file = args.file

   if not name:
      print('--name is required')
   if not content and not file:
      print('--name or --file is required')

   if file:
      content = open(file, 'r').read()
   content = compress(content.encode()) # content type will be byte after compress
   hash = sha256(content)
   size = len(content)
   chunk = 10
   for i in range(0, size, chunk):
      res = requests.post(f"{server_url}/objects/{name}", files={
                                                            'obj': content[i:i+chunk],
                                                            'hash':hash,
                                                            'version': args.version,
                                                            'start': i,
                                                            'length': chunk,
                                                            'size': size})
      print(i, chunk, size, res.status_code)

我们传递Object的content，可以藉由—content指定内容，或是—file指定档案位置

• Download Object

python frontend.py --type do --name obj01 --version 1.7.3

介绍完type的意思後，就直接上Download Object的程序码了

def downloadObject(args):
   name = args.name
   version = args.version
   if not name:
      print('--name is required')
      return
   if not version:
      print('--version is required')
      return
   # 1. get object size
   size = requests.get(f"{server_url}/objects/size/{name}?version={version}").json()['size']
   print(f'Size of Object: {size} bytes')
   content = b''
   start = 0
   # download object until length of content fits size
   while len(content)<size:
      try:
         content += requests.get(f"{server_url}/objects/{name}?version={version}&start={start}").content
      except KeyboardInterrupt:
         sys.exit()
      except Exception as e:
         start = len(content)
         print('Something wrong, downloading again...', e)
   try:
      print(f'Get content: {decompress(content).decode()}')
   except Exception as e:
      print(f'Something wrong: {e}')

由於前端需要知道我要download的这个object何时要结束，所以需要先request object size，再根据size大小判断何时结束

• Version List of an Object

python frontend.py --type vl --name obj01

这部分比较简单，就是一个简单的GET request取的metadata的资料，读者可以自行理解

• Location where Object store

python frontend.py --type ol --name obj01 --version 1.7.3

这部分比较简单，就是一个简单的GET request取的metadata的资料，读者可以自行理解