Python的shelve.open可以以嵌套的方式调用吗？

Question

我正在尝试编写一个使用搁置来持久存储返回值的记忆库。如果我有记忆函数调用其他记忆函数，我想知道如何正确打开架子文件。

import shelve
import functools

def cache(filename):
    def decorating_function(user_function):
        def wrapper(*args, **kwds):
            key = str(hash(functools._make_key(args, kwds, typed=False)))
            with shelve.open(filename, writeback=True) as cache:
                if key in cache:
                    return cache[key]
                else:
                    result = user_function(*args, **kwds)
                    cache[key] = result
                    return result
        return functools.update_wrapper(wrapper, user_function)
    return decorating_function

@cache(filename='cache')
def expensive_calculation():
    print('inside function')
    return

@cache(filename='cache')
def other_expensive_calculation():
    print('outside function')
    return expensive_calculation()
other_expensive_calculation()

但这不起作用

$ python3 shelve_test.py
outside function
Traceback (most recent call last):
  File "shelve_test.py", line 33, in <module>
    other_expensive_calculation()
  File "shelve_test.py", line 13, in wrapper
    result = user_function(*args, **kwds)
  File "shelve_test.py", line 31, in other_expensive_calculation
    return expensive_calculation()
  File "shelve_test.py", line 9, in wrapper
    with shelve.open(filename, writeback=True) as cache:
  File "/usr/local/Cellar/python3/3.4.1/Frameworks/Python.framework/Versions/3.4/lib/python3.4/shelve.py", line 239, in open
    return DbfilenameShelf(filename, flag, protocol, writeback)
  File "/usr/local/Cellar/python3/3.4.1/Frameworks/Python.framework/Versions/3.4/lib/python3.4/shelve.py", line 223, in __init__
    Shelf.__init__(self, dbm.open(filename, flag), protocol, writeback)
  File "/usr/local/Cellar/python3/3.4.1/Frameworks/Python.framework/Versions/3.4/lib/python3.4/dbm/__init__.py", line 94, in open
    return mod.open(file, flag, mode)
_gdbm.error: [Errno 35] Resource temporarily unavailable

对于这类问题，您有什么建议?

Answer 1

不，您可能没有嵌套具有相同文件名的shelve实例。

shelve模块不支持对搁置对象的并发读写访问。(多个同时读访问是安全的。)当一个程序打开一个架子供写时，其他程序不应该打开它供读或写。Unix文件锁定可以用来解决这个问题，但这在不同的Unix版本中是不同的，并且需要了解所使用的数据库实现。

https://docs.python.org/3/library/shelve.html限制

Answer 2

而不是尝试嵌套调用打开(正如您已经发现的那样，不起作用)，您可以让您的装饰器维护对shelve.open返回的句柄的引用，然后如果它存在并且仍然打开，则重用该句柄用于后续调用:

import shelve
import functools
def _check_cache(cache_, key, func, args, kwargs):
    if key in cache_:
        print("Using cached results")
        return cache_[key]
    else:
        print("No cached results, calling function")
        result = func(*args, **kwargs)
        cache_[key] = result
        return result
def cache(filename):
    def decorating_function(user_function):
        def wrapper(*args, **kwds):
            args_key = str(hash(functools._make_key(args, kwds, typed=False)))
            func_key = '.'.join([user_function.__module__, user_function.__name__])
            key = func_key + args_key
            handle_name = "{}_handle".format(filename)
            if (hasattr(cache, handle_name) and
                not hasattr(getattr(cache, handle_name).dict, "closed")
               ):
                print("Using open handle")
                return _check_cache(getattr(cache, handle_name), key, 
                                    user_function, args, kwds)
            else:
                print("Opening handle")
                with shelve.open(filename, writeback=True) as c:
                    setattr(cache, handle_name, c)  # Save a reference to the open handle
                    return _check_cache(c, key, user_function, args, kwds)
        return functools.update_wrapper(wrapper, user_function)
    return decorating_function

@cache(filename='cache')
def expensive_calculation():
    print('inside function')
    return

@cache(filename='cache')
def other_expensive_calculation():
    print('outside function')
    return expensive_calculation()
other_expensive_calculation()
print("Again")
other_expensive_calculation()

输出:

Opening handle
No cached results, calling function
outside function
Using open handle
No cached results, calling function
inside function
Again
Opening handle
Using cached results

编辑:

您还可以使用WeakValueDictionary来实现装饰器，这看起来更容易读:

from weakref import WeakValueDictionary
_handle_dict = WeakValueDictionary()
def cache(filename):
    def decorating_function(user_function):
        def wrapper(*args, **kwds):
            args_key = str(hash(functools._make_key(args, kwds, typed=False)))
            func_key = '.'.join([user_function.__module__, user_function.__name__])
            key = func_key + args_key
            handle_name = "{}_handle".format(filename)
            if handle_name in _handle_dict:
                print("Using open handle")
                return _check_cache(_handle_dict[handle_name], key, 
                                    user_function, args, kwds)
            else:
                print("Opening handle")
                with shelve.open(filename, writeback=True) as c:
                    _handle_dict[handle_name] = c
                    return _check_cache(c, key, user_function, args, kwds)
        return functools.update_wrapper(wrapper, user_function)
    return decorating_function

一旦一个句柄没有其他引用，它就会从字典中删除。因为我们的句柄只有在对修饰函数的最外层调用结束时才会超出作用域，所以当句柄打开时，字典中总是有一个条目，而在它关闭后，字典中没有条目。

Answer 3

您打开了文件两次，但从未真正关闭它以更新文件。最后使用f.close ()