HEX
Server: Apache
System: Linux www3.pit.tblive.com 5.14.0-687.38.1.el9_8.x86_64 #1 SMP PREEMPT_DYNAMIC Wed Aug 12 17:19:12 EDT 2026 x86_64
User: awaldron (1020)
PHP: 8.1.34
Disabled: exec,passthru,shell_exec,system
Upload Files
File: //opt/cloudlinux/venv/lib/python3.11/site-packages/ssa/modules/__pycache__/storage.cpython-311.pyc
�

?Ocj�'���dZddlZddlZddlmZddlmZmZmZm	Z	ddl
Z
ddl
mZmZm
Z
ddlmZmZdZdeefd	�ZeGd
�d����Zdeefd�Zd
�Zde	eeeffd�Zd�ZdS)zF
Represents storage where ssa data is collected, stored and extracted
�N)�	dataclass)�List�Iterator�Tuple�Dict)�func�cast�distinct)�
session_scope�
RequestResulti@B�returnc�.�g}dx}}d}t|��D]g\}\}}|dz}|�||kr|}|�||kr|}|tkr|�|���Btjd|��}|tkr|||<�h|tkr
||d<||d<|S)ai
    Collect a per-path durations list bounded to ``MAX_URL_DURATION_SAMPLES``.

    Below the cap every duration is kept (order preserved). Above it, reservoir
    sampling (Algorithm R) keeps a uniform random sample so percentile/mean
    estimates stay statistically representative without letting one hot path
    materialise an arbitrarily large list.
    Nr����)�	enumerate�MAX_URL_DURATION_SAMPLES�append�random�randint)	�	dur_group�	reservoir�true_min�true_max�total�seen�_�duration�idxs	         �J/opt/cloudlinux/venv/lib64/python3.11/site-packages/ssa/modules/storage.py�_sample_durationsr s����I���H�x�
�E�(��3�3�*�*���m�q�(��q�����x�(�2�2��H���x�(�2�2��H��*�*�*����X�&�&�&�&��.��D�)�)�C��-�-�-�!)�	�#���
�'�'�'��	�!�� �	�"�
���c�D�eZdZUeed<eeed<eed<eed<dS)�
DomainData�domain_name�domain_total_reqs�is_a_wordpress_domain�urls_numberN)�__name__�
__module__�__qualname__�str�__annotations__r�int�bool�r!rr#r#?sF������������C�y� � � ����������r!r#c#�zK�t|��5}|�tjt	jdtj��t	jtj��t	j	tj
��t	jttj
�������tjt	jdtj�����tjt	jdtj����}t!j|d����}|D]^\}}t%|��}d}dgdz}|D]*\}	}
}}}
||t'|
��<t||
��}�+t)||||���V��_	ddd��dS#1swxYwYdS)z7
    Iterates data from database domain-by-domain.
    �%Hc��|dS�Nrr/��items r�<lambda>z#iter_domains_data.<locals>.<lambda>Xs
��VZ�[\�V]�r!��keyr�)r$r%r&r'N)r�queryr�domainr�strftime�
created_at�Count�id�max�	wordpress�countr
�path�group_by�order_by�	itertools�groupby�tupler-r#)�engine�db�results_by_hour�results_by_hour_groupedr$�group�domain_results_by_hourr'�requests_number_by_hourr�hour�requests_num�is_wordpress�urlss              r�iter_domains_datarTGs�����
�v�	�	��"��H�H��$��
�d�M�$<�=�=��
�=�+�,�,����0�1�1��
�8�M�$6�7�7�8�8�
�
��X�m�*�D�M�$�
�@X�,Y�,Y�
Z�
Z�
�X�m�*�D�M�$�
�@X�,Y�,Y�
Z�
Z�	�#,�"3�O�I]�I]�"^�"^�"^��"9�	�	��K��%*�5�\�\�"��K�()�c�B�h�#�=S�
5�
5�9��4��|�T�5A�'��D�	�	�2�!�+�t�4�4����'�"9�&2�'�	���
�
�
�
�	���������������������s�FF0�0F4�7F4c
#� K�t|��5}|�tjt	jdtj��t	jttj	tj�����d��t	j
tj���d��t	jttjtj�����d�����tj|k���tj�|�����tjt	jdtj�����tjt	jdtj����}d}dgdzdgdzdgdz}}}|D]q\}	}
}}}
|r/||	kr)|t+||||���fV�dgdzdgdzdgdz}}}||t-|
��<||t-|
��<|
|t-|
��<|	}�r|	t+|	|||���fV�ddd��dS#1swxYwYdS)	z6
    Iterates urls data from database url-by-url.
    r1�url_throttled_reqs�url_total_reqs�
url_slow_reqsNrr9�rCrVrWrX)rr:rrCrr<r=�Sumr	�hitting_limits�
sqlalchemy�Integer�labelr>r?�is_slow_request�filterr;�in_rDrE�dictr-)rIr$�	all_pathsrJ�	urls_data�
previous_pathrVrWrXrCrP�url_throttled_req�
url_total_req�url_slow_reqs              r�iter_urls_datarims�����
�v�	�	�-
�"��H�H��"��
�d�M�$<�=�=����m�:�J�<N�O�O�P�P�V�V�Wk�l�l��
�=�+�,�,�2�2�3C�D�D����m�;�Z�=O�P�P�Q�Q�W�W�Xg�h�h�
�
��V�M�(�K�7�
8�
8�
�V�M�&�*�*�9�5�5�
6�
6�
�X�m�(�$�-��m�>V�*W�*W�
X�
X�
�X�m�(�$�-��m�>V�*W�*W�
X�
X�	��
�=>�C�"�H�q�c�B�h�QR�PS�VX�PX�M�N��JS�	!�	!�F�D�$�)�=�,��

a��$�!6�!6�!��*�+=�'5�&3�	�������FG�C�"�H�q�c�TV�h�YZ�X[�^`�X`�M�N�"�,=��s�4�y�y�)�(5�N�3�t�9�9�%�'3�M�#�d�)�)�$� �M�M�
���#5�-�+�	
�
�
�
�	
�	
�	
�K-
�-
�-
�-
�-
�-
�-
�-
�-
�-
�-
�-
����-
�-
�-
�-
�-
�-
s�I$J�J�
Jc#�K�t|��5}|�tjtj���tj|k���tj��}tj	|d���}|D]\}}|d�|D��fV��	ddd��dS#1swxYwYdS)zA
    Get information about durations of requests url-by-url.
    c��|dSr3r/r4s rr6z#get_url_durations.<locals>.<lambda>�s
��d�1�g�r!c��g|]\}}|��Sr/r/)�.0rrs   r�
<listcomp>z%get_url_durations.<locals>.<listcomp>�s��:�:�:�[�Q���:�:�:r!N)
rr:rrCrr`r;rErFrG)rIr$rJrd�durations_by_pathr8rMs       r�get_url_durationsrp�s����
�v�	�	�
;�"��H�H�]�'��)?�@�@�
�V�M�(�K�7�
8�
8�
�X�m�(�
)�
)�	�&�-�i�9M�9M�N�N��+�	;�	;�J�C���:�:�E�:�:�:�:�:�:�:�:�	;�
;�
;�
;�
;�
;�
;�
;�
;�
;�
;�
;�
;����
;�
;�
;�
;�
;�
;s�BB7�7B;�>B;c
#�K�t|��5}|�tjt	jdtj��t	jttj	tj�����d��t	j
tj���d��t	jttjtj�����d�����tj|k���tjt	jdtj�����tjt	jdtj����}|�tjtj���tj|k���tj��}t+j|d����}t+j|d����}t/|d��}t/|d��}|��|��|\}	}
|\}}|	|kr�d	gd
z}
d	gd
z}d	gd
z}|
D]>\}}}}}||
t1|��<||t1|��<||t1|��<�?t3|��}|	|t5|	|
||���fV�t/|d��}t/|d��}n'|	|krt/|d��}nt/|d��}|�|��ddd��dS#1swxYwYdS)u
    Stream per-URL data for a single domain.

    Yields ``(path, durations, url_data)`` per path, where:
    - ``path`` is the URL path (str);
    - ``durations`` is a list of raw request durations for that path;
    - ``url_data`` is a dict with per-hour aggregates
      (``url_throttled_reqs``, ``url_total_reqs``, ``url_slow_reqs``
      as 24-element lists) plus ``path``.

    Merges two ``ORDER BY path`` streams in lock-step instead of
    materialising the per-domain ``{path: [durations...]}`` dict up
    front, which caused OOM on large databases (CLPRO-3077). A second
    benefit: this drops the redundant ``WHERE path IN (...)`` clause —
    both queries already filter by ``domain``, so the path list adds
    no selectivity, only memory pressure on the SQL parser.
    r1rVrWrXc��|dSr3r/��rs rr6z&iter_domain_url_data.<locals>.<lambda>��
���1��r!r7c��|dSr3r/rss rr6z&iter_domain_url_data.<locals>.<lambda>�rur!Nrr9rY)rr:rrCrr<r=rZr	r[r\r]r^r>r?r_r`r;rDrErrFrG�nextr-r rb)rIr$rJ�	agg_query�	dur_query�agg_by_path�dur_by_path�agg_item�dur_item�agg_path�	agg_group�dur_pathrrVrWrXrrP�	throttledr�slow�	durationss                      r�iter_domain_url_datar��sD����$
�v�	�	�=3�"��H�H��"��
�d�M�$<�=�=����m�:�J�<N�O�O�P�P�V�V�Wk�l�l��
�=�+�,�,�2�2�3C�D�D����m�;�Z�=O�P�P�Q�Q�W�W�Xg�h�h�
�
��V�M�(�K�7�
8�
8�
�X�m�(�$�-��m�>V�*W�*W�
X�
X�
�X�m�(�$�-��m�>V�*W�*W�
X�
X�	�
�H�H�]�'��)?�@�@�
�V�M�(�K�7�
8�
8�
�X�m�(�
)�
)�	� �'�	�~�~�F�F�F���'�	�~�~�F�F�F����T�*�*����T�*�*���"�x�';�"*��H�i�"*��H�i��8�#�#�&'�S�2�X�"�"#��r���!"��b��
�7@�4�4�3�A�t�Y��t�4=�&�s�4�y�y�1�05�N�3�t�9�9�-�/3�M�#�d�)�)�,�,�-�i�8�8�	����%�+=�'5�&3�	���	�	�	�	� ��T�2�2����T�2�2����H�$�$�
 ��T�2�2�����T�2�2��G�"�x�';�5=3�=3�=3�=3�=3�=3�=3�=3�=3�=3�=3�=3����=3�=3�=3�=3�=3�=3s�L%M�M�M)�__doc__rFr�dataclassesr�typingrrrrr\rr	r
�ssa.dbrrrr-r r#rTrir+rpr�r/r!r�<module>r�sq��������
�
�
�
�!�!�!�!�!�!�.�.�.�.�.�.�.�.�.�.�.�.�����+�+�+�+�+�+�+�+�+�+�/�/�/�/�/�/�/�/�%�� �D��I� � � � �F�����������#��*�!5�#�#�#�#�L1
�1
�1
�h;�d�3��c�
�?�.C�;�;�;�;�"O3�O3�O3�O3�O3r!