You can have a Run Configuration to run your celery workers which then allows you to debug simply by clicking the debug button. Here is how I set that up in PyCharm 2020.1

Module name: celery

Parameters: worker --workdir=/path/to/your_project_dir/ --app=project_name.celery --loglevel=INFO

You need to set up a remote python interpreter and then set other configs like the image above.