Fixing stuff

This week I’m fixing various ingest errors. Turns out that MongoDB doesn’t like dots (‘.’) in field (key) names. I can hazard a guess why, but the fix isn’t all that pretty. In the former MongoDB faq there used to be a suggestion for fixing this, but the page seems to be replaced. Using python I replace the dots with unicode characters for ‘fullwidth fullstop’ char U+0FFE. To fish out the keys with dots in them from the JSON schema:

for key, value in schema_dict['items']['properties'].items():
    if '.' in key:
        new_key = key.replace('.', '\uff0e')
        schema_dict['items']['properties'][new_key] = schema_dict['items']['properties'].pop(key)
required = schema_dict['items']['required']

for required_val in required:
    if '.' in required_val:
        required.append(
            required.pop(
                required.index(required_val)
            ).replace('.', '\uff0e')
        )

I don’t know if this is quite idiomatic python, but it’s terse and functional enough for me.

I’m betting on not being any text fields having a dollar sign (‘$’) in front of them, since this is reserved for MongoDB keywords…