You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
We have found the default state management mechanics of modular diffusers to lead to a few issues and questions:
It is easy for a block to mutate a state variable that represents user input which means later blocks cannot distinguish between user input value vs. block modified value for a state variable. Fixes to denoising step list cache init and conditioning change cache init #137 describes a scenario where denoising_step_list was used to represent user input value, but it was also modified by a block which broke logic for checking if the user input value was different from the current value being used by the pipeline. We also currently are not clearly distinguishing between user inputs and intermediate inputs that come from other blocks. See Fixes to denoising step list cache init and conditioning change cache init #137 for a note on how modular diffusers is supposed to be able to handle a intermediate_inputs property for blocks, but it doesn't seem to work right now.
It is not clear what the convention should be for tracking state that is specific to the block, but that is not relevant for other blocks. For example, the EmbeddingBlending block currently tracks "private" state in the same global PipelineState and just uses an underscore prefix eg _transition_active to denote that it is private. This is okay for now, but seems accident prone and ideally private state is truly encapsulated in a block instead of leaking out to global PipelineState.
It is easy to fallback to adhoc access of a state variable even if it is not declared as an input/intermediate input in the block as is done here. Ideally, someone can look at the declared inputs/intermediate inputs and know that these are the only values being accessed in the block. But, the ability to do this adhoc access of a state variable breaks that expectation.
I'm opening up this discussion thread to track ideas for improving how we use modular blocks and specifically to address the problems I see above.
An initial idea is to create a class that inherits from ModularPipelineBlocks that we could use for all of our block definitions. This class would then provide safer access, with conventions we define, to state so that a developer creating a block definition does not have to think about them.
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
Uh oh!
There was an error while loading. Please reload this page.
-
We have found the default state management mechanics of modular diffusers to lead to a few issues and questions:
denoising_step_listwas used to represent user input value, but it was also modified by a block which broke logic for checking if the user input value was different from the current value being used by the pipeline. We also currently are not clearly distinguishing between user inputs and intermediate inputs that come from other blocks. See Fixes to denoising step list cache init and conditioning change cache init #137 for a note on how modular diffusers is supposed to be able to handle aintermediate_inputsproperty for blocks, but it doesn't seem to work right now._transition_activeto denote that it is private. This is okay for now, but seems accident prone and ideally private state is truly encapsulated in a block instead of leaking out to global PipelineState.I'm opening up this discussion thread to track ideas for improving how we use modular blocks and specifically to address the problems I see above.
An initial idea is to create a class that inherits from ModularPipelineBlocks that we could use for all of our block definitions. This class would then provide safer access, with conventions we define, to state so that a developer creating a block definition does not have to think about them.
Beta Was this translation helpful? Give feedback.
All reactions