U uie@sddlZddlZddlZddlZddlmZmZmZmZddl m Z ddl m Z m Z ddlmZmZddlmZmZmZddlmZmZmZddlmZmZmZed eZd d d hZd ddhZ e!dddZ"e#dddZ$e!e%dddZ&e!e'e'dddZ(e!dddZ)e!e!dd d!Z*d"d#Z+e!e!e!d$d%d&Z,ej-d'd(gd)d*d+d)Z.ej-d,d(gd-d*d.d-Z/ej-d/d(gd0d*d1d0Z0ej-d2d(gd3d*d4d3Z1ej-d5d(gd6d*d7d6Z2dS)8N) Blueprintrequestjsonify current_app)bulk_db)require_bulk_api_keyget_access_token_for_alias)parse_publish_at_iso_to_tscreator_can_post_now) safe_filenamevideo_duration_seconds_ffprobebuild_public_media_url)process_one_scheduled_jobresolve_access_token_for_jobrefresh_submitted_job_status)query_creator_infoupload_video_direct_postfetch_post_statusZbulk_apicompletefailed cancelled scheduled submitting submitted) table_namec Csb|d|d|}g}|D]:}z||dWq"tk rZ||dYq"Xq"|S)z Devuelve lista de columnas reales del schema para inserts robustos. Funciona con row_factory sqlite3.Row o tuplas. zPRAGMA table_info()name)executefetchallappend Exception)currrowscolsrr&5/var/www/html/luxverbi-app/app/blueprints/bulk_api.py_table_columnssr()jobcs||}t|d}t|fdd|D}d|}ddgt|}d|d|d}||t| |S) z^ Inserta en bulk_jobs filtrando por columnas reales (evita 'N values for M columns'). bulk_jobscsi|]\}}|kr||qSr&r&).0kvcolsetr&r' 3sz$_insert_bulk_job..z, ?zINSERT INTO bulk_jobs (z ) VALUES (r) cursorr(setitemsjoinkeyslenrtuplevalues)connr)r"r$filteredcolnames placeholderssqlr&r.r'_insert_bulk_job+s r?)errreturncssdStdr"dSdr0dSdddddd d d d d ddddddddddg}tfdd|DrvdSdsdrdSddddd d!d"g}tfd#d|Drd$Sd$S)%u Decide si el fallo merece reintento (requeue) en lugar de marcar el job como failed. Filosofía: - Reintentar todo lo que sea probablemente transitorio: 5xx, 429, internal_error, timeouts, errores de red, "cannot_post_now", etc. - Marcar failed solo para errores claramente permanentes (fichero inexistente, privacy inválida, duración excedida, etc.) Tcannot_post_nowZmissing_publish_idzhttp=429ztoo many requestsZ rate_limitZthrottlezhttp=500zhttp=502zhttp=503zhttp=504Zinternal_errorzservice unavailablez bad gatewayzgateway timeouttimeoutz timed outZ readtimeoutZconnecttimeout connectionzconnection resetZ temporarilyz try againc3s|]}|kVqdSNr&)r+m)er&r' [sz&_is_transient_error..direct_post_failedcreator_info_failedZfile_not_found_on_serverduration_exceeds_maxinvalid_privacy_levelbranded_cannot_be_self_onlymissing_privacy_level missing_videomissing_filenamec3s|]}|VqdSrE) startswith)r+p)r@r&r'rHlsF)strlowerrQany)r@Ztransient_markersZpermanent_prefixesr&)rGr@r'_is_transient_error<sV      rV)r@ attempts_nextrAcCsVd}|p ddrdnd}t||dtd|d}t|td d }t||S) u Backoff exponencial con jitter, con cap. Evita martillear la API. - cannot_post_now: arranca más alto. - cap: 6h (el sistema sigue intentando, pero sin bucle agresivo). i`TrB<rrgg?)rQminmaxintrandomuniform)r@rWcapbaseexpZjitterr&r&r'_backoff_secondsrs rd)rAcCsz|dp d}Wntk r,d}YnX|rtk r|}z tddt|ddfWYSd}~XYnX|stdd d dfSd tj krtdd d dfStj d }|j stdd d dfStjdpd}tjdpd}|stddd dfStjdddk}tjdddk}tjdddk}tjdddk} tjdddk} tjdddk} tjdddk} | r|dkrtddd dfSz t |} Wn@tk r}z tddt|ddfWYSd}~XYnXt | \}}|s&tdd|ddfS| dp4g}||krTtdd |d!dfS| d"d#krhd}| d$d#kr|d}| d%d#krd}| }| }| }t jd&}t|j }ttd'|}tj||}||| d(}t|}t|tr\|d)kr\|d)kr\||kr\zt|Wntk rDYnXtdd*||d+dfSt|t jd,t jd-t jd.d/}zVt||||||| | | d0|d1 }|d2pid3}td#|||||pdd4d5fWStk r}z tdd6t|ddfWYSd}~XYnXdS)7u Publish inmediato (server-to-server). Aquí SÍ es correcto generar video_url firmado, porque se publica en el momento (la ventana de TTL es suficiente). Soporta: account_alias (recomendado) o access_token. rerXrfFalias_token_failedokerrordetailsNZ%missing_access_token_or_account_aliasrrvideorOrPtitle privacy_levelrN allow_comment1 allow_duet allow_stitchcommercial_toggle0brand_organic_togglebrand_content_toggleis_aigc SELF_ONLYrMrJrB)rrreasonprivacy_level_optionsrL)rroptionscomment_disabledT duet_disabledstitch_disabled UPLOAD_DIR_max_video_post_duration_secrrK)rrdurr]PUBLIC_BASE_URLMEDIA_SIGNING_SECRETMEDIA_TOKEN_TTL_SECONDS)stored_filenamepublic_base_urlsigning_secret ttl_seconds PULL_FROM_URL) rfcaptionrdisable_comment disable_duetdisable_stitchrrrmode video_urldatart)rrtr init_resprrerI)rrformgetrirr!rrSfilesfilenamerr rconfigr r^timeospathr5saver isinstanceremover r)rerfrGfilerrrrrrrrr creator_info can_post_nowrrrrr upload_dir original_fnr save_pathmax_durrrrrtr&r&r'rs .   .      *   z/api/bulk/statusapi_bulk_statusc Csttjdpd}tjdp(d}|r~z t|}Wn>tk r|}z tddt|ddfWYSd}~XYnXtjd pd}|r|stdd d dfSzt ||}td |d dfWStk r}ztdt|d dfWYSd}~XYnXdS)zJ Status (server-to-server). Soporta account_alias o access_token. rerXrfFrrrNrtZ"missing_access_token_or_publish_idrT)rrr) rrrrrirr!rrSr)rerfrGrtrr&r&r'rRs  . z/api/bulk/find_existingapi_bulk_find_existingcCsttjdpd}tjdp4tjdp4d}|sPtddddfS|sftdd ddfSt|}ttj d ||d }td d i|dfS)u Lookup server-to-server para deduplicación por alias + nombre de archivo original. El scheduler externo puede usar `can_schedule`: - True => no hay job activo bloqueando. - False => ya existe uno en curso para ese video/cuenta. rerXrzrFZaccount_alias_requiredrrZmissing_original_filename BULK_DB_PATHrerzrTr) rrrrrirr rrr)reZoriginal_filename_rawrzlookupr&r&r'rms  z/api/bulk/scheduleapi_bulk_schedulecCsttjdpd}|s0tddddfSz t|}Wn>tk rz}z tddt|ddfWYSd }~XYnXtjd pd}|stdd ddfSz t |}Wn>tk r}z tdd t|ddfWYSd }~XYnXd tj krtddddfStj d }|j s4tddddfStjdpDd}tjdpZd}|sxtddddfStjdpd }|dkrtddddfStjdddk} tjdddk} tjdddk} tjdddk} tjdddk} tjdddk}tjdddk}|rN|d krNtdd!ddfSt jd"}t|j }tt jd#||d$}d }|dkr|d%}n|d&kr|d%p|d'}|rt|d(}td)d)|tkrd*nd+|d,|d(|d-|||d. d/fStj}|d0|}tj||}||tt}||||d1d2d ||||| rRd3nd2| r^d3nd2| rjd3nd2| rvd3nd2| rd3nd2|rd3nd2|rd3nd2||dd d d |d4}tt jd#}t||||td)||||d5d/fS)6uM Schedule (server-to-server). Para jobs programados, account_alias es requerido para que el cron/worker pueda refrescar tokens con refresh_token. Importante: - NO generamos video_url firmado aquí, porque caduca (TTL) antes de publish. - El worker debe generar video_url justo en el momento de publicar. rerXFZ)account_alias_required_for_scheduled_jobsrrrrN publish_atmissing_publish_atZinvalid_publish_atrrOrPrrrN dedupe_mode active_only>allnonerZinvalid_dedupe_moderrrrrrrrrrrMrrrr{rr|rqTZalready_scheduledZalready_existsrwrr) rZskippedrrwrqrrrzrerrrrrr)rp created_at_tsrrnext_attempt_tsrqattempts last_errorrfrerrrrrrrrrrrzrrtrurvrs)rrwrrrre) rrrrrirrr!rSr rrrTrrr rror~uuiduuid4hexrrr5rr^rrr?commitr})rerfrGrrrrrrrrrrrrrrrrZexisting_lookupZexisting_matchZexisting_statusrwrrnow_tsr)r:r&r&r'rs  . .                     z/api/bulk/process_dueapi_bulk_process_duec Cstttjddpd}tt}ttjddp:d}tt j d}| }dddddd}| d||f| }td t|d g}g}t} |D]4} t| } | | kr|| q| | || q|r.|dkr.||} |D]} | d | || d |fq|d t|7<||D]} | d }| d||f|jdkr`q2|z4t| t j dt j dt j dt j dd\}}}WnBtk r}z"d}dt|jdt|}W5d}~XYnX|r0|}t| }| d|||||f|jdkr$|dd7<|q2|p8d}t| dd}t|r|t||}| d|||||f|jdkr|dd7<n0| d||||f|jdkr|d d7<|q2| d!|f| }|D]} | d }t| \}}|sq|d"krB|d#kr&d$nd%}| d&|||||fn| d'||||f|jdkrr|d(d7<|q|t d)||d*d+fS),u Cron endpoint: - Auth: X-Api-Key - Procesa jobs due (scheduled) y refresca jobs submitted. Mejoras: - Reintento con backoff para errores transitorios (incl. 5xx/internal_error). - Cooldown por cuenta para no disparar 2 publicaciones seguidas en segundos. - Protección anti-atasco: si el worker lanza excepción, requeue en lugar de dejar "submitting". max_jobs10ZBULK_MIN_SECONDS_BETWEEN_POSTSZ180rr)scheduled_submittedscheduled_failedscheduled_requeuedsubmitted_updatedscheduled_deferred_by_cooldownz SELECT * FROM bulk_jobs WHERE status='scheduled' AND next_attempt_ts <= ? ORDER BY publish_at_ts ASC LIMIT ? z DEBUG: Found z due jobsz UPDATE bulk_jobs SET next_attempt_ts=?, updated_at_ts=? WHERE id=? AND status='scheduled' AND next_attempt_ts <= ? rprz UPDATE bulk_jobs SET status='submitting', updated_at_ts=? WHERE id=? AND status='scheduled' rrrrr)rrrrFzworker_exception: z: Na UPDATE bulk_jobs SET status='submitted', publish_id=?, attempts=attempts+1, last_error=NULL, last_status=NULL, last_fail_reason=NULL, access_token=?, next_attempt_ts=?, updated_at_ts=? WHERE id=? AND status='submitting' rZ unknown_errorra UPDATE bulk_jobs SET status='scheduled', attempts=?, last_error=?, next_attempt_ts=?, updated_at_ts=? WHERE id=? AND status='submitting' rz UPDATE bulk_jobs SET status='failed', attempts=?, last_error=?, updated_at_ts=? WHERE id=? AND status='submitting' rz} SELECT * FROM bulk_jobs WHERE status='submitted' ORDER BY updated_at_ts ASC LIMIT ? )PUBLISH_COMPLETEZFAILEDZSEND_TO_USER_INBOXrrrz UPDATE bulk_jobs SET status=?, last_status=?, last_fail_reason=?, updated_at_ts=? WHERE id=? AND status='submitted' z UPDATE bulk_jobs SET last_status=?, last_fail_reason=?, updated_at_ts=? WHERE id=? AND status='submitted' rT)r processedrr)!rr^rrrrrenvironrrrr2rrprintr7r3rmr addrrowcountrr!type__name__rSrrVrdrr}r)rrZ cooldown_sr:r"rZdueZpickeddeferredseenrjkeyZbump_tsr%rwrmsgZ _init_resprGrtZused_access_tokenr@rWZnext_tssubsrqZ fail_reasonZ new_statusr&r&r'rs         ,                 )3rrrr_flaskrrrrdb.bulkr services.authrrservices.creatorr r services.mediar r r Zservices.bulk_workerrrr tiktok_clientrrrrbprr~rSr(dictr?boolrVr^rdrmrorxrrouterrrrrr&r&r&r's@    6/ }