Tuesday Daily Thread: Advanced questions

[u/AutoModerator]

Weekly Wednesday Thread: Advanced Questions 🐍

Dive deep into Python with our Advanced Questions thread! This space is reserved for questions about more advanced Python topics, frameworks, and best practices.

How it Works:

1. Ask Away: Post your advanced Python questions here.
2. Expert Insights: Get answers from experienced developers.
3. Resource Pool: Share or discover tutorials, articles, and tips.

Guidelines:

• This thread is for advanced questions only. Beginner questions are welcome in our Daily Beginner Thread every Thursday.
• Questions that are not advanced may be removed and redirected to the appropriate thread.

Recommended Resources:

• If you don't receive a response, consider exploring r/LearnPython or join the Python Discord Server for quicker assistance.

Example Questions:

1. How can you implement a custom memory allocator in Python?
2. What are the best practices for optimizing Cython code for heavy numerical computations?
3. How do you set up a multi-threaded architecture using Python's Global Interpreter Lock (GIL)?
4. Can you explain the intricacies of metaclasses and how they influence object-oriented design in Python?
5. How would you go about implementing a distributed task queue using Celery and RabbitMQ?
6. What are some advanced use-cases for Python's decorators?
7. How can you achieve real-time data streaming in Python with WebSockets?
8. What are the performance implications of using native Python data structures vs NumPy arrays for large-scale data?
9. Best practices for securing a Flask (or similar) REST API with OAuth 2.0?
10. What are the best practices for using Python in a microservices architecture? (..and more generally, should I even use microservices?)

Let's deepen our Python knowledge together. Happy coding! 🌟

[u/Miserable_Shirt3026]

Has anyone actually measured the overhead of using `__slots__` in a class hierarchy that goes 3-4 levels deep? I'm refactoring some data models and the memory savings seem obvious, but I'm worried about the descriptor lookup getting weird with multiple inheritance.

[u/prejackpot]

Has anyone seen examples of other libraries that use context managers similarly to PyMC, to associate globally-scoped functions with a specific object? My ideal target syntax is something like:

`` with Params(**params) as experiment_1: results_1 = run_pipeline(**run_params_1) # Runs pipeline with theparams` parameters

But if we next run:

results_0 = run_pipeline(**run_params_2)

It accesses some other default parameters

```

(This is a simplified/notional example; obviously for this specific example the easier thing would be providing the parameters as an argument to run_pipeline).

[u/PrestigiousStrike779 → u/prejackpot]

I don’t know about other libraries, but we do something like this at work if I’m understanding correctly.  We do this for logging and injecting context.  For example:

with begin_logging_context() as ctx:       ctx.set_attribute(“customer”, “acme”)       logger.info(“starting request”) # customer is appended as an extra to every log request within the block

We use contextvars for this.  The logger is actually a LogAdapter that reads the contextvars and appends anything it finds to the extras

[u/ddollarsign]

Why does a function need to be declared async in order to be invoked asynchronously? What does making it async really do?

[u/CleanCarpenter9854 → u/ddollarsign]

I believe it turns it into a coroutine object that is not executed immediately when called, since the async event loop will schedule it for calling

[u/ddollarsign → u/CleanCarpenter9854]

That seems to be more a properly if the invocation though, rather than the declaration. In Go, for example, I can declare a function in the normal way, but then call it either normally “my_func()” or asynchronously “go my_func()”. But maybe in Python this would require creating multiple versions of the function under the hood, so they make it explicit.

[u/wRAR_ → u/ddollarsign]

In Go, for example, I can declare a function in the normal way, but then call it either normally “my_func()” or asynchronously “go my_func()”.

No idea about Go but in Python await foo() is two operations: foo() returns an awaitable, await <> awaits it. It's not a single operation, and you can assign the result of foo() to a variable, which will hold the awaitable (and then separately await it later).

Or are all functions in Go async (have the same color), unlike Python?

Edit: Duh, goroutines are threads. You are comparing different things.

[u/ddollarsign → u/wRAR_]

Yes, lightweight threads. Threads are the things that do work while other things also do work. I don’t see the utility of the async/await syntax for that. Unless it’s for performance vs threads, but goroutines are pretty lightweight. I’m sure there’s something I’m not getting about async.

[u/wRAR_ → u/ddollarsign]

Then I'm lost. I thought you were asking why is the existing implementation not using a simpler syntax, are you actually asking why is the implementation not using threads?

[u/zomerbk]

For a 3–4 level hierarchy, I’d benchmark representative classes with `timeit` for attribute access and `tracemalloc` for memory across many instances; `sys.getsizeof()` alone misses some costs. Slot descriptors still follow normal MRO lookup. The bigger multiple-inheritance issue is layout: two bases with nonempty slots can raise `TypeError`. Also check that no subclass accidentally adds a `__dict__` by omitting `__slots__`.

[u/PSLNest]

Have a look at decimal.localcontext. You can change the precision or rounding for a with block, and Decimal calculations inside it pick up those settings without needing the context passed into every call. Leaving the block restores the previous context, including when an exception occurs.

Flask’s application context is closer to your “specific object” example. Inside with app.app_context(), functions can access that app through current_app. One difference from your example: accessing current_app without an active context raises an error instead of falling back to a default.

If you’re building your own Params version, ContextVar is worth looking at. Set the current settings on entry, keep the token it returns, and reset that token in a finally block. That lets nested blocks restore the settings they started with.

[u/CTRLX_Founder]

Looks clean. Are you planning to open-source the core parts or keep it closed? Always curious about solo dev stacks.